Downside is that you are limited to what software your distro has vetted and provides. With a strong and trusted app sandbox, I can more easily trust less-vetted software, because I can see what it can do before running it.

E.g. if it can only do file accesses outside of it's own settings folder after prompting me, I know that it has way less abilities to screw up my system.

I don't think that can replace a package manager for "complex" or "infrastructure" software, but for other things it could open the selection up. Many people already run sandboxed applications, in sandboxes called "Firefox" or "Chrome".