A capability model fixes this: the application launches a file chooser which it does not control, which can then return it an fd or other token giving the application the capability to read from or write to a specific file.

Of course, this would be a considerable effort. But it's probably where we're going.

I've long argued for applications to use file choosers they do not control, but not for security reasons. I've wanted it for UI reasons.

Somewhere I've got a screenshot of a Linux system I had with half a dozen applications trying to open a file. Each used a different GUI framework or widget toolkit, each of which had its own file dialogs, and so the user gets presented with half a dozen different file choosing UIs. It was completely ridiculous.

Making the UI suck less was not a good enough reason to get any of the major GUI frameworks to support separating file choosing from apps. Maybe improving security will be a good enough reason.

Major GUI frameworks agree with you and have already adopted this approach and it is already used by many applications, and this has already been the case for years.

Flatpak already does this. It is called portal. The grandparent post is wrong.

The problem is what haplens when the program does not use standard file choosers, file APIs, etc.

Flatpak in theory vs flatpak in practice are different things, as you highlight yourself ("...does not use standard file choosers, file APIs, etc.").

Suddenly, it's now on app developer to support the packaging system and not the other way around.

I mean what do you want to have happen? ptrace every application, stop on every openAt(2) call and pop up a permissions dialog?

macOS can get away with solutions like this because they control the kernel and userspace but on Linux it’s a much tougher problem.

And also because macOS has only one desktop environment :)

TBF you could put support into the two common GUI toolkits, to defer to a Flatpak portal.

But that’s exactly what GTK and QT have! It’s what to do with apps that don’t use some abstraction that supports portals that’s the issue.

Well, there’ll always be a long tail. But if you start with the popular apps and frameworks first, you get the most bang for your buck.

It wouldn't surprise me however if sandboxing is only a good model for GUI not console app.

IIRC the Windows 10X approach will be to put all non-sandboxed apps in the exact same environment.

As mentioned above, Flatpak is already doing the two most popular toolkits. Programs just don't use toolkits to open files. So there is not a lot of "buck" here, and the tail is rather long.

Can't the sandbox override the corresponding syscalls? E.g. open() and readdir() which then block until the user has selected the file? This means you'll have to click the file twice (once for the sandbox and once for the application) but at least it is more secure.

Ideally, not even getting the FD, but access to the data only (decrypted in flight)?

Of course, in "classic" apps, there is explicitly no protection. They don't promise it, and you don't get those by default when trying to install a snap.