As an immigrant, this is one of those things that still perplexes me about Americans despite my having grown up here. Nobody in Bangladesh thinks they should care as much about people in Africa as they do about their own neighbors. I see that attitude in the US, but it seems to mostly be an excuse not to really care about anybody except at the most diffuse level.
Additionally, if government officials are supposed to be governing the nation for the benefit of all mankind rather than looking after the interests of the people whom they theoretically represent, we might as well dispense with the pretense of representative government and try to formulate some other basis for state legitimacy.
It's not as if we don't have our own poor --blacks and whites.
Let's also remember, up and till WWi we were not a rich country. We were quite rural and not all that excellent. Many of today's poor countries were not that far behind the US.
Saying the civil war US was an example of a great army is like saying Syria has the greatest armies in the ME just because so many people are armed.
The US was rural, technologically backward, compared to some european powers, and not that far ahead of others.
Some of the issue comes down to fairness.
We set the rules. We have not achieved WH Bushes thousand points of light new world order single government. We look out for ourselves just as China, Mexico, Singapore do.
And as the most powerful, we cannot stand by and pretend everybody else is on their own. That is the most irresponsible thing any country can do. Even Poland helps their neighbors unconditionally in time of need. And they're no better off.
That said, to be fair to all potential immigrants, we should grant all of them equal potential access (i.e. someone from South Africa having the same chance to legally ingress into the US as a Canadian or Mexican who are right next door --it might also take into account their population, so someone form Indonesia has the same odds as someone from Belize.
And, to be fair to the American population, we should ensure those who come here are here to fill gaps in our society --so ensuring they don't undercut Americans jobs (that is let meat packers have to pay $25/hr to get locals to do that job rather than say have them say that no American wants to do that job. I'd bet many poor whites as well as many poor blacks would take those jobs at those wages.
Also, any underdeveloped population which moves to the US (or Kuwait or Chile) will typically increase their own per capita consumption (from a global perspective) also if they come from a country wallowing in economic mismanagement (say Albania) allowing Albanians (or Georgians) to flee means even less stability for those locales as it takes their motivated people and transplant them into a place that doesn't need them as much. There are many incidental things which a policy can affect. We're not a nation of 120 million or 200, we're at 330 and climbing and most with a voracious appetite for consumption (including the fast learning newcomers).
That said, fairness in immigration should be a priority rather than just allow those who runt the gauntlet successfully. And in addition, tie it to the ease of Americans's ability to migrate into these other countries, if we choose.