> It is no different in the sense that rape and assault (e.g. punching or kicking someone) are essentially similar physical acts

Correct and this is reflected by the fact that both are criminal acts. If the latter is particularly severe it might even carry a penalty even greater than rape. This example doesn't really do a good job making the case for why sex work harms participants or society as a whole, because unlike battery paying people who face starvation and homeless to do work is not only acceptable it's seen as totally normal. Save for people who have enough wealth live off for the rest of their lives, everyone is working to avoid homelessness and starvation.