The above is a good perspective to keep in mind. Conventionally-raised foods cost less, because every corner is cut to get that food as cheaply to you as possible. Many long-term costs are not included. Paying twice as much for organics may seem outrageous, but it is a matter of perception. The question should not be, "Why are these organic foods so expensive?" but rather: "How terrible must they be treating the earth and these animals to get this food so cheap?"
Conventionally-raised foods would not be so cheap if they showed their true cost in terms of environmental degradation, ecosystem destroyal, and carbon emissions (that does not even include animal cruelty, for which you just can't assign a price). The day when farmers are required to compensate for the carbon emissions that somebody else will have to clean up and pay for all the destruction their chemicals cost down river is the day when organics will be the cheapest on the block.
Simply having the label "organic" is not always the best, though. The problem is twofold: "Certified organic" has rigid standards, but that does not mean the food is naturally raised. In contrast, there are no standards for "naturally raised", so just about any producer can use that label with impunity. For example, you can buy "certified organic" eggs, but the chickens could be locked into cruel cages and simply fed a diet of organic grain instead of conventional grain. Those are certainly not eggs I would want, even if they qualify as "organic". I want eggs from chickens who forage in fresh, green areas. Its not just a matter of animal cruelty. Pasture-raised are superior nutritionally, as well.
It gets more complicated. I have no problem eating something that is "organically raised" (but not certified). This is what you would get if you dug out a patch of dirt in the city and grew your own crops free from any pesticide. However, just being in the city with runoff from nearby roads and the potential of non-organic chemicals already in your soil would prevent you from meeting the criteria for organic certification. However, this is just fine with me (as long as you are not planting on the grounds of an extinct factory or something). Organic certification measures not only the chemicals the farmer uses, but also considers any second-hand exposure from the soil, air or water that could have been polluted generations ago. Sometimes this is an unnecessary extreme for the average consumer (although it would be a worthy goal for all food to meet this standard, someday).
As you can see, "Organic" is sometimes too extreme and sometimes inadequate, as shown above.
When in doubt, its still a good thing to buy organic. But if you can research a little more, that is even better. The most important thing for me is that farmers do not add any artificial pesticides or fertilizers to the land, practice soil conservation and that their animals are raised in the most healthy and natural way possible. If their farms have some minor contamination from up-river that is not as important to me as this other criteria.
Besides, if farmers are ever going to convert conventional farms to organic ones, it would be good if consumers would cut them some slack. Sometimes it takes years for the soil to lose all trace of the chemicals that were once used there. As long as they are not actively using those chemicals anymore, I will support them.
The most excellent website is Eat Wild. You can search for farmers and vendors in your area that supply pasture-raised meat, milk and eggs. There are also great articles on that website about the benefits of eating food raised this way. The list of benefits is substantial, from your health to the environment. Shop your local farmers markets for fresh fruits and vegetables, when possible. The vendors are usually more-than-willing to tell you exactly how they raised their foods.