And sometimes they would be right; speaking for myself, I've worn baggy clothes for a long time, but at some point I finally managed to lose some weight and got down below 90 kilos, and was able to go down a size pants and shirts (size L), which fit and looked a lot better. Great confidence boost at the time, didn't change anything for my er, relationship prospects though.

It's WFH time now though, I'm wearing 2/3XL shirts (good cut though, they don't look baggy) and pajama pants and have done so for over a year now. I am here for the comfort.

"Vanity sizing" is a real issue in clothing. Sizes were always abstractions but now measurements are an abstraction as well.

Indeed!

https://en.wikipedia.org/wiki/Vanity_sizing