Maybe my judgement or efficiency is bad when I try to help my neighbor. Okay, whoopsie. Now apply that margin of error to a foundation whose decisions impact millions of people and possibly entire societies, possibly for generations. The unintended or possibly negative effects can be enormous and long-lasting.
And even the poorest parts of the US are doing much better than a lot of poor countries in Africa and Asia.
Operating at a huge scale requires you to lump people together into groups and make assumptions about who they are and what they deserve, as you've done in your example. To me that sounds antithetical to the concept of charity. And even with the best intentions, if you mess up, you're messing up a huge scale.
It seems that our notion of charity has been warped in such a way that it somehow doesn't count unless it's some enormous large-scale mission carried about by Charity, Inc for the benefit of some group of people far away. It's all very abstract, and that just creates room for fraud or exploitation.
Paying off a local person's debts or putting their child through college will probably be far more impactful and meaningful than sending your money off to some giant organization.
How do you judge the actions of someone when those actions are powerful enough to move markets, take down regimes, and change people's lives for generations?
We take them at their word and assume that everything they do is well-intentioned and good and has zero negative impact or secondary effects, but is that really the case?
To me it seems like the only charity that can be trusted is a small-scale one that acts locally and with lots of transparency.