If you don’t have 2FA enabled for your account, you are in the least secure position compared with enabling any 2FA method.
Either way, there’s no need to use SMS for 2FA with Google, they support many other forms of 2FA.
Real 2FA would (theoretically) never make your account less secure than 1FA, because even if the second factor has 0 security, it shouldn't decrease the security of the first factor.
However, it is true that this may not always be the case for imperfect implementations, like your example. I can aldo imagine that social engineering might have a higher succes ratio if the intruder can say "it really is me! I have the correct second factor, I just lost my first factor...".
What do you suppose happens when the hot guy in the club who bought them and all their friends a round of drinks says they can make a month's salary in ten minutes if they allow a few of his mates to skip that "annoying" ID check?
And I'm sure that afterwards, once the carrier figures out who was responsible, they'll get fired. But meanwhile your accounts were taken over and leveraged for some scam or (if you own "cool" things like a short Twitter handle or that rare drop in a video game) sold to the highest bidder.
at the end of the day someone is sitting at a computer and given an id-card from a stranger and clicking the "yes, i verified their id card".
Whereas, of course, the reason we're discussing SIM attacks isn't that some bad guys might use your monthly download credits, or make a phone call that you pay for (the phone company would of course undo these charges once it was apparent that their employee caused the problem) but instead that this can be leveraged to gain access to all sorts of accounts your phone company has no responsibility for whatsoever.
The security was totally appropriate for the scope in which it was intended to be used, but that doesn't make it magically appropriate for all possible scopes.