Sweatshops: A violation of human rights or the way out of poverty?

I saw a thought-provoking article by New York Times columnist, Nicholas D. Kristof that was reproduced in today's Star's business section. His main point:

The central challenge in the poorest countries is not that sweatshops exploit too many people, but that they don’t exploit enough.

I remember coming across this idea back in my class on Developmental Economics. It was shocking then, and after a few years, it's still shocking now. Shocking because it makes so much sense. We have been brainwashed to think that sweatshops are the axis of evil and that we should boycott all goods made in such conditions. But what are the alternatives?

Kristof gives an example of families in Phnom Penh that live on 'a mountain of festering refuse, a half-hour hike across, emitting clouds of smoke from subterranean fires.' These families, including little kids, dig through the rubbish for recycleable products that they sell for pennies. They have to cope with garbage trucks that may run them over, leaving them dead or dismembered, in addition to the hellish conditions on the dump. To them, a sweatshop is paradise - at least they are working under sheltered conditions. And yet we are denying them this "priviledge" by boycotting sweatshop goods?

True, none of us would choose to work in a sweatshop, but then again, we've never experienced worse conditions. If a sweatshop was actually a step up from your current living conditions, woulnd't you want to work there too? But surely there's a middle ground, some kind of compromise. The low wages in poorer countries are always the big draw for these companies, but they could also provide better, more humanitarian working conditions. Something that would appease the human rights activists, and yet still make it worthwhile for the companies to have these "sweatshops" that are actually ladders out of poverty. 

Am I being too romantic? Maybe Nike comissioned Kristof to write that article...