That's not the problem with SMS for 2FA. The problem with SMS for 2FA is that cell phone accounts were never intended to serve as what amounts to a high security authenticator service, and cell phone companies are resistant to this newfound 'responsibility', somewhat understandably so.

By default, someone can call up your cellular provider, claim to be you, pass trivial to no security questions, and request a replacement SIM be mailed out, or that your number be ported to another device. Or they can slip a bit of cash to any employee who works at any cell phone store that sells service for your carrier.

SMS 2FA isn't better than just a password. It's objectively worse, dramatically increasing the attack surface. Compromise someone's cell phone account and you are virtually guaranteed access to their bank/retirement/investment accounts, email, social media, etc. And they are virtually powerless to do anything about it for at least a few hours while they scramble to, say, get phone service working again and rush to contact everyone they can think of. By the time you're able to even get to your bank to talk to a branch manager and show all sorts of proof of identity, your accounts could be long since cleaned out.

Some providers finally are offering secondary passwords for porting/SIM replacement, that sort of thing. Absolutely call them and request your account be locked down as much as possible, ask to specify a secondary password, etc.