Authy backups are encrypted with a password (at least, if you set them to), so you can't gain access with just the phone number.

However I believe their "OneTouch" system (e.g. Twitch & Namecheap) is linked to phone numbers only, so that could be breached via SMS.

I'd recommend using a standalone 2FA app which can be backed up to your own cloud storage. (e.g. Google Drive/Dropbox)