That is a strawman. I never said everyone who uses classes perfectly uses semantic elements.
My point is that if you are using <div class=my-element> you don't have to change your .my-element CSS selector or JS selection code to improve your code to <p class=my-element>. If you are using <my-element> it is much more work to change your selectors and now you have two ways of doing things depending on if you are using a native semantic element or a div (either a tag selector or class selector). You have made your styling code depend on your element choice which makes it harder to change.
You could say the same about divs. I’ve seen pages where everything is a div. No paragraphs, no headings, no tables, no lists, just divs.