A connection exposes you to surveillance, and yes, it's worse with SIMs, but having WiFi on just adds a little latency to the same thing. Public Routers that you're discovering do the same thing. I had thought this was the entire point of iOS devices randomizing their MACs when scanning available networks (no idea if this ever got shipped with Android devices).
I am probably in the "nutty" quadrant when it comes to concerns about tracking, surveillance concerns, and online privacy, but I also don't really like needlessly inconveniencing myself just to undermine the entire purpose of why the inconvenience is necessary in the first place. Your phone is always talking -- SIM card or not. I'm not even sure modern non-smart phones (flip revivals and such) are wifi free.
Such protections are a good ideal, but a lot of people have an unrealistic idea of the threat actors they're trying to protect against.
Tracking is not an issue, if you keep your phone in airplane mode (wifi and bluetooth can be enabled on most phones, but mobile network stays disabled), or if you just remove the sim card, and put it back, on those rare occasion when you need it (so you don't have to bother other people)