For example, blanket access to filesystem seems bad. Start with, most apps get access to some particular subdir of $HOME, like say "Documents", their own config for and systemwide files (not user-specific). Then you have a directory of home which you know is more vulnerable, but the rest of it, like .ssh, is protected. Select programs like ssh or rsync can have access.

Perhaps a simpler starting point is a runner script that enforces such rules on an opt-in basis from the user, and tooling that makes it easy to run binaries through this. Like a reverse sudo sort of thing.

I'm certainly not saying it's easy, but I'm not sure it is impossible either. It's a bit like an internal firewall.

Even if apps request blanket permissions, that's a win already. On mobile, when apps do it, it is a big red flag for me and I skip.