Just because you can sideload doesn’t mean apps can just be unconfined; if you sideload an Android app for example, it’s still restricted by the OS sandbox and still has to prompt you for access to contacts, location, etc.

And as an aside: you can sideload apps on Android. Have you seen any banks try to convince users to sideload their app? I sure haven’t.