I think the progressive label comes from two angles: there’s a lot of history of religious moralizing and policing around this which these days identifies strongly with conservatism (note especially how that often shifts blame from the patrons to the workers), and that starts with the idea that there’s something uniquely degrading about this in way that there isn’t about, say, physical work which might leave you damaged for life or jobs which are basically lying or harassing strangers. It might be interesting to think about exactly why you think that’s worse than, say, working in a meat processing plant with a high rate of injury or being a collections agent for a hospital.

The second is having empathy for the worker, a key progressive position. That says, for example, that if you do think that there’s something truly degrading about the work the answer is to have economic and social policies which reduce the number of desperate people without good alternatives, or to have better regulation of the middlemen profiting from the situation, etc. rather than traditional approaches focused on punishing workers.