I generally agree, but one problem is that many of Apple's rules are enforced primarily or exclusively through policy, not actual technical permissions. This is especially true for all the high-level ones around tracking.

Obviously, things like phone book access can be solved for sideloaded apps just as well as for App Store ones – just like Android has been doing for many years.