The main worry was never about it "replacing a worker". It's about one worker + GPT replacing two or three, because of the increased productivity gains you mentioned.
I've seen it happening in some fields already. First with translation, now with copywriting and GPT-4. Hiring was frozen for those areas for a while in some companies I'm familiar with.
Fyi, GPT 5 won't be coming soon and GPT 4 vs. 3.5 is not that different in terms of quality.