There's probably no conspiracy, just generalized classism.
It takes a lot of empathy (something that is actively selected against in our corporation dominated workforce) to bypass that.
Add into that the gravity of money (ie, rich get richer) and political corruption (money buys power) and you have a basic framework for classism.
If I want to farm your labor, I need to find you, make sure you are going to work long enough at a site for me to make money, and I have to hope that you wont do something that will sour the relationship I have with "my" customer (the business I am renting you to.)
The margins for staffing places are actually pretty small per person, so its not so much unjustifiable as it is dehumanizing.
I worked temp out of college in the dot com bust. I lived out of a borrowed car. I was desperate for work and had relevant skills, but many temp agencies wouldn't touch me. It was stupid and dehumanizing, but it was also pretty unjustifiable.
So maybe that is the logic, but it's specious logic.