Moral culpability assumes responsibility. I'm racist and sexist because I was born in a racist and sexist society -- that was not my fault. I spread disease because germs exist, and that's not my fault. But now that I've learned to see and recognize racism, sexism and germs, I try to fight them and not to spread them. If I don't wash my hands or do not fight sexism and racism, then I am morally culpable now, but those facts in themselves do not imply moral culpability even though they do imply harm (in all three cases).

Your statement about exploitation is just an expression of laziness and anger. It is hard for me to believe that you believe that scholars believe that the existence of a group of people is immoral. If that is what you believe, then you aren't paying attention.