For a normal user with an app that doesn't allow export, it's more or less equivalent to a yubikey. There's no reasonable way to get the secrets out of a locked phone or a yubikey, but an expert with a lot of resources probably could. And a normal user isn't reinstalling their phone OS so that worry's out of the way.

For apps that do allow export/transfer, it's up to the user if they want to keep the secret safe. But really at some point it's up to the user no matter what method or hardware you use.

And since most sites don't have an easy way to have multiple tokens, allowing export/transfer also reduces the temptation to insecurely store a copy of enrollment QR codes.