And a mildly technical angry ex is a lot less likely than phishing. These are valuable topics but people go way way way too far and say that SMS is horrible and should be basically banned while TOTP is fabulous and a completely viable alternative, which is just fantasy.
Depends on your threat model, but unlike SIM swapping this may not be out of the reach of even a mildly technical angry ex.