It's definitely not just you. And I really get sick of the people saying things like "just zoom" or "set a custom minimum font size".

When I read that, I read, "we're ableist and here's a few crumbs so we can feel better about ourselves".

Do you also think every site should come with a built in screen reader? Accessibiliy should be about enabling everyone to use the website - using additional tools if needed - not to dumb it down to the lowest common denominator. If zoom / min font size breaks things complain about that, otherwise don't force giant fonts on everyone else.