Being able to run some applications from super fast but small NVMe disk and less important ones from slow but huge spinning rust disk, being able to run applications from a network share, CD, USB stick, etc. Desktop OSs have been able to do these sorts of things since the 1980s without all the hoops Linux Desktop software makes you jump through to do them.
> Being able to install and run flatpaks from any directory seems like it’d be difficult to implement for little benefit.
If Flatpak had considered this ability from the beginning it wouldn't be. Hell it might not be that difficult now, I haven't looked at the source for Flatpak and how it does things behind the scenes is otherwise not documented as far as I can tell. As you mentioned, AppImage implements it without much difficulty, and AppImages can be securely isolated by firejail. Combining the two concepts in the same package shouldn't be that hard. Keep runtimes in "installations" and let applications exists as single-directory self-contained units wherever the user wants them. What is fundamentally difficult about this?
> unnecessary since AppImage already offers exactly that functionality
If AppImage were fast becoming the new standard in Linux Desktop application distribution we wouldn't be having this conversation.
Linux desktops can do this. Flatpak maybe can't (Idk, I don't use them that often, but I'd be surprised if they couldn't), but even before AppImage, Flatpak, Snap, etc you could always just run an application from wherever you want. You could do that 20 years ago, and you can do it today.
Desktops in the 80s didn't have sandboxing, but modern Linux desktops do. You can either implement sandboxing yourself through syscalls, use something like Systemd, or use the significantly more user friendly Flatpaks.
...Or don't use sandboxing at all, and do whatever you want. Download a (hopefully statically-linked) ELF from the internet, chmod +x it, then execute.
> standard in Linux Desktop application distribution
There is no such thing. There never has been, and there never will be :P
What Guix and Nix give you are a package manager that can be used with a signature distro or installed upon another distro, and that package manager makes an effort to be the last package manager you'll need. Both of them are packaging the normal types of programs you'd expect, but also emacs packages, vim plugins, python packages, rust packages, go packages, and so on. Nearly every guix upgrade I do shows me that a bunch of new r-foo packages (R-lang stuff) got packaged. The way I see it, with enough time and manpower, these package managers will have just about everything you'd ever need (exceptions for things like proprietary software, mainly in the case of guix, although these can and do still exist in unofficial channels). Even in their current state they are good enough at doing this that I'm able to get all my emacs packages from the package manager and not use elpa/melpa at all. This also gives you rollbacks for free. If an emacs plugin breaks, you can roll back to a previous generation, and optionally do a partial upgrade to then re-upgrade everything except that broken package. This process is basically instant as it's just moving some symlinks around in the store. You can also have multiple versions of programs installed without issues, and until you do garbage collection, you likely do by default.
Declarative package manager is also cool even as a regular user. If you choose to use a guix manifest to manage your user's packages, you have a good visual of everything you've intentionally installed on your system (not cluttered up with the deps they pulled in), and you can easily add/remove them with a text editor + re-running a command afterward to reapply the manifest.
I don't consider myself a programmer, and I have not built packages for most distros (I have made a guix package or two locally with some help, but never submitted them upstream), so I thought my perspective as someone using these tools may be valuable. I wouldn't say I ever learned the nix lang or guile to any sort of level of fluency. I basically treat a configuration.nix or config.scm the same way I treat my sway config, neovim config, and so on. The syntax is a little complicated, but the community is there to help, and there's also the documentation. I just know enough to get by as a regular user and I still greatly appreciate guix.
> Flatpak is gaining ground, which is better than continuing with the current paradigm as far as I'm concerned, but it is still worse than the simple way things were done in saner desktop operating systems.
(and from your other comment:)
> Application management on Linux is already complicated enough without having to learn a new language to use it effectively.
Linux is the only system (maybe now FreeBSD, if they've completed the migration to a packaged base system yet) where every single piece of software on a complete and usable system can be managed uniformly with a single set of automatic tools. You are absolutely correct that people using such a system are loathe to return to the fundamentally unmanageable anarchy of a system like Windows, where
* every installer does basically whatever the hell it wants;
* you can never really be sure that an uninstaller does its job;
* much software simply cannot be installed or uninstalled non-interactively;
* managing non-interactive installation for apps which DO support non-interactive management is often achievable only through some bespoke process;
* every single fucking program feels entitled to litter your system with its own bespoke auto-update mechanism (sometimes one that runs in the background constantly!);
* a brand new system with virtually no desired application software takes up 20-40 GiB, instead of 4-8 GiB;
* it is difficult to even *discern* whether or not the whole system is up to date, and bringing a system up to date requires several manual processes;
* even in the year of our lord two-thousand and twenty one, whole system installs mysteriously become slow after a few years so that they must either be reinstalled (the only sure-fire way) or treated with a laborious, manual cleanup process just to achieve sane, normal performance again.
Flatpak isn't a 'step in the right direction' which will eventually take us to the manual, repetitious, chaotic, redundant mess of managing software on other operating systems. It's a fundamental improvement over that model, because it retains
* very good deduplication of shared runtimes via OSTree;
* easy management by automatic tools;
* conventional and technical constraints on what applications can do to set themselves up;
* a single central interface for software distribution;
* a more complete specification of the dependency chain, unlike the implicit and unspecified dependencies on various Windows components from different eras that cause the base Windows installation (even without any applications) to grow and grow and grow.
> > > standard in Linux Desktop application distribution
> > There is no such thing. There never has been, and there never will be :P
> Did I mention the other thing I hate about Linux Desktop? It's community.
This is how you react to playful commiseration about the difficulty of coordinating unification between independent, distributed, democratically managed volunteer projects? Please, I urge you in the strongest possible terms: continue using Windows exclusively, forever and ever. :)
Only the software that has been painstakingly packaged for that particular version of that particular distribution can be managed by said tools. This is one of its biggest flaws and part of the reason Flatpak even exists.
You rightly describe a lot of problems with the installer mechanism of traditional Windows, which is not what I am advocating for[0]. I'm advocating for something more like original Mac applications, or RiscOS AppDirs, or NeXT/current MacOS Application Bundles, or just folders with files in them in DOS, or you know, AppImage. Single object applications that can be copied, deleted, and moved the same as any file without a bunch of special tooling. I contend that the need for special tooling is to work around an overly complicated model, not inherent to the problem space.
> This is how you react to playful commiseration about the difficulty of coordinating unification between independent, distributed, democratically managed volunteer projects? Please, I urge you in the strongest possible terms: continue using Windows exclusively, forever and ever. :)
When someone asks me about my use case for features for the sole purpose of being pedantic and telling me things I already know that are entirely useless and that I have heard variations of for twenty goddamn years while literally using an emoji that represents a human face sticking it's tongue out... well let's just say I'm not inclined to have a high opinion of them. Frankly, there's good reason Linux is in a distant third place as a Desktop OS, despite how much Windows sucks, and it isn't because it provides a seamless experience and its community is so helpful and welcoming.
[0] It's weird that when you criticize something in Linux Desktop, for some reason its proponents always go all whataboutism on Windows.
Yes, that's true. This is a pretty marginal problem for F/OSS-oriented end users of distros with large package repositories, and that's probably why it's not a problem that gets the kind of focus you'd like to see within the Linux community. My needs are well-served by the package management model, and I don't use very much proprietary software, so for me personally, it feels very worth it to package whatever new hotness I need on my system and retain the management benefits of the package manager.
But 98% of end users (even fairly technical end users) don't want to package any software themselves, and it's not really possible for third parties to do so properly with proprietary software. (Although you can find many examples of shoehorning proprietary binaries into normal-ish packages in the AUR and in Nixpkgs.) The fact that ‘Linux’ is not really an OS platform is a real problem for publishers who might want to release software on the Linux desktop.
> You rightly describe a lot of problems with the installer mechanism of traditional Windows, which is not what I am advocating for[0]. I'm advocating for something more like original Mac OS applications, or RiscOS AppDirs, or NeXT/current MacOS Application Bundles, or just folders with files in them in DOS, or you know, AppImage.
AppImage has one big problem, which is that it basically just assumes but doesn't specify a fairly large base runtime. This is a serious portability problem, and it's why AppImages are not usable in their intended way on systems like NixOS and GuixSD, and instead require a wrapper/launcher. Flatpak doesn't share this problem because its design involves a launcher program that can be wrapped or provided with a special environment, and it's designed to more fully encode the runtime dependencies of apps packaged for it. (In principle, Snap is the same way, but I doubt it'll ever be packaged in Guix due to the proprietary server-side code and I just don't think there's much interest in it on the Nixpkgs side.) This could be solved by standardization, but I have to agree that that is very unlikely to happen.
Of the examples you give, macOS ≥10 application bundles are the only ones I've really used. Maybe this is nitpicking, but I don't think macOS .apps are a great model because
* they still leave us with the same update management problems as I outlined in the Windows case
* they don't really cover everything application packages or installers are expected to be able to do, like bundle kernel modules, modify the operating system configuration, or set up runtimes for programming languages (this is why you end up with executable scripts that run as root for macOS .pkg installers)
* AppBundles can and do have dependencies which are unmanaged
* real applications often involve multiple bundles, which is one of a few reasons that your tools again have no real uninstall capabilities (take a look at a few Homebrew Cask formulae[0] and you'll see examples of this, e.g., here[1])
* it's not hugely important, but AppBundles do still result in enormous application/installer downloads, and a base macOS install takes up 2-4x the space it probably could (this is less egregious than on Windows because at least Macs come with a fairly comprehensive set of basic GUI applications)
Something like Flatpak can address the problems with AppBundles in a way that clones like AppImage can't.
That said, this:
> Keep runtimes in "installations" and let applications exists as single-directory self-contained units wherever the user wants them. What is fundamentally difficult about this?
basically sounds okay to me and I don't see why it couldn't be done. But there are two problems (for me, at least):
* I don't really like the idea of Flatpak applications being distributed as files becoming popular, because I'd rather developers submit their apps to Flathub or somewhere similar so that they're still usable via remote management tools;
* and where to draw the line between the runtimes and the application is not clear, since in Flatpak, runtimes can depend on other runtimes. The more you deduplicate, the more you lose the ability to manually manage your application storage. The more AppBundle-like you make Flatpaks, the less deduplication and internal dependency management you can do.
> It's weird that when you criticize something in Linux Desktop, for some reason its proponents always go all whataboutism on Windows.
You mentioned that Windows is the desktop operating system you're currently using, so it seems Windows' application management norms must be acceptable to you. I thought about including a section about the case on macOS (similar to the one in this comment), but I didn't do so because you mentioned Windows and not macOS in your original comment here.
—————
0: https://formulae.brew.sh/cask/
1: https://github.com/Homebrew/homebrew-cask/blob/HEAD/Casks/ad...
> * they still leave us with the same update management problems as I outlined in the Windows case
Not necessarily, there are a lot of ways to go about discovering which applications need updates and where to get them. Off the top of my head: they have a manifest file (or header) that specifies a URL for acquiring the update and the OS maintains a list of paths for every application ever launched to be checked with a single command, as well as optionally being able to search for applications to update.
> * they don't really cover everything application packages or installers are expected to be able to do, like bundle kernel modules, modify the operating system configuration, or set up runtimes for programming languages (this is why you end up with executable scripts that run as root for macOS .pkg installers)
Agreed, but I'm mostly concerned with desktop applications, if that wasn't clear. Such add-on software needs some mechanism of being easily visible to the OS, which probably necessitates something like a fixed path, but I don't think it can't still be self-contained. IIRC Slax did something with AUFS that involved self-contained objects in a special directory that were applied as overlays over the base system at boot and on-demand, which is one way to do it. I think all Haiku packages kinda work like that too.
> * AppBundles can and do have dependencies which are unmanaged
I don't think they should. Or rather, the dependencies that are not part of the base system are considered part of the application and should be contained with it and updated with it. UI toolkits, system interfaces, cryptography, networking, and other such libraries used by a large number of applications should be part of the base system. As you mentioned, part of the problem here is that Linux Desktop has never had a concrete idea of the separation between platform (base system) and application. However, Flatpak kinda does, because it has 'runtimes'. If Flatpak managed the runtimes as it does today, but allowed self-contained portable application folders to run from wherever and use said runtimes, that'd be pretty close to ideal I think.
Out of order because related: >> * I don't really like the idea of Flatpak applications being distributed as files becoming popular, because I'd rather developers submit their apps to Flathub or somewhere similar so that they're still usable via remote management tools;
Whereas I don't like the idea of developers and users being beholden to the whims of a repo to be able to distribute and consume software conveniently. We may just have to agree to disagree on that one.
>> * and where to draw the line between the runtimes and the application is not clear, since in Flatpak, runtimes can depend on other runtimes. The more you deduplicate, the more you lose the ability to manually manage your application storage. The more AppBundle-like you make Flatpaks, the less deduplication and internal dependency management you can do.
It shouldn't be that hard to find a reasonable balance I don't think. Some duplication isn't really a problem because, as I mention elsewhere, assets are a heck of a lot bigger these days than any binary. Further, many (some would say most) "shared" libraries are only really used by a single application. There are obvious candidates for the 'base system'/'runtime' platform: UI, system interface, networking, cryptography, etc. The problem there with Linux of course is that ABI compatibility is frequently broken in some of these so you'd have to have a bunch of versions to cover everything.
> * real applications often involve multiple bundles, which is one of a few reasons that your tools again have no real uninstall capabilities (take a look at a few Homebrew Cask formulae[0] and you'll see examples of this, e.g., here[1])
See above. To reiterate: I think this is just a poor way to do things. If you have some tightly integrated set of applications they should be managed as a single unit. Think an 'MS Office' application instead of a separate Excel, Word, and Outlook.
> * it's not hugely important, but AppBundles do still result in enormous application/installer downloads, and a base macOS install takes up 2-4x the space it probably could (this is less egregious than on Windows because at least Macs come with a fairly comprehensive set of basic GUI applications)
I don't really use a modern mac and haven't since the early 2000s, but as far as I can tell from using applications in general the actual binary portion of them is usually dwarfed by things like graphics and sound assets, which isn't helped by the traditional package management system anyway. Besides, we're living in an era when people voluntarily bundle an entire goddamn web browser to handle their application's GUI. There's no way native-GUI AppBundles are worse.
> You mentioned that Windows is the desktop operating system you're currently using, so it seems Windows' application management norms must be acceptable to you.
I guess that's one way you could word it. Another one is that Windows does a lot of things wrong, and its application distribution mechanism isn't ideal, but overall the things Windows does wrong don't bother me as much as the things Linux does wrong. As I mentioned in the very first post in this thread though: Windows is rapidly become so bad that Linux might soon be the least-worst choice. When that day happens I will not be any more inclined to like the way Linux Desktop does things, I'll just finally hate Windows more.
I generally like the centralization and vetting associated with Linux distro repos, but this would be a decent approach for desktop applications that at the very least wouldn't be annoying to use or deal with.
> part of the problem here is that Linux Desktop has never had a concrete idea of the separation between platform (base system) and application
Yeah. I kind of like that Linux is this way but I do see how it is also a problem.
> If Flatpak managed the runtimes as it does today, but allowed self-contained portable application folders to run from wherever and use said runtimes, that'd be pretty close to ideal I think.
I think that could be pretty nice. It would certainly be user-friendly, and it would still allow Linux distros to vary and experiment in all of the ways that they like to vary and experiment.
> we're living in an era when people voluntarily bundle an entire goddamn web browser to handle their application's GUI. There's no way native-GUI AppBundles are worse.
lmao true
> IIRC Slax did something with AUFS that involved self-contained objects in a special directory that were applied as overlays over the base system at boot and on-demand, which is one way to do it. I think all Haiku packages kinda work like that too.
There's a research distro called Distri that does some tricks like this, and I think the design is really good. It doesn't allow hooks and triggers (which I think you'd like) and distributes packages as SquashFS images. It does some union mounting stuff (and I think user namespace trickery) to handle shared dependencies.
https://michael.stapelberg.ch/posts/2019-08-17-introducing-d...
There's also GoboLinux, which rethinks the shared global paths in a Mac-ish way that's pretty neat: https://www.gobolinux.org/
Neither is usable as a daily driver or has a sizeable community or anything like that (of course lol).
I do think that for some things, we do need to rethink the FHS, and if we want something that publishers can deal with, we need a clear separation between base system and end user applications on Linux. (We also need that for users who don't want to be system administrators, and who will be surprised and upset that installing what they think of as end user applications can result in dependency conflicts. See the LTT ‘do as I say!’ debacle.)
Imo, some day it would be great to have something like Guix or Nix generate a base system that is made available in what Nix/Guix folks would call an ‘impure’ way, and then for applications to be installable on top via something like Flatpak for desktop applications and an old-fashioned ports system for developer tools. The complexity of tools like Guix and Nix makes sense for distro makers, and sharing dependencies is actually valuable on a base system where you want tight integration. But then publishers and users who don't want to deal with that complexity can use distribution mechanisms like we've discussed, and the door to the packaging system is open for people who want to intervene in the base system instead of just building on top of or alongside it.
I think we will probably get there, very, very slowly.
1TB NVME drives aren't very expensive on the order of $80-$100 US. This is already big enough for your entire OS even with several very large games installed. The natural split if you have multiple drives would be between OS and backups/multimedia.
I see this immediately being used for pirated software and games. It could also be used to implement a cross platform app store like interface to install legitimate software. It could even be integrated with the same store interface as one uses to install traditional packages and flatpaks for a true one stop shop.
That isn't the point. I have in my possession several small form factor computers that have 16GB disks in them. "just spend $80-100 each on them" is not a good excuse for having such an inflexible system that it demands everything be on one disk.
The original Mac did this on floppies in 1984. It is 2021, our software should be able to do better.
- caching in ram
- on disk cache
- an explicit read cache device
All of these are broadly applicable and effective to the other 99.9% of actual devices.
It's neat old Mac's from the 80s could to do that...but they had to be able to do that in order to be useful. In other words, it wasn't a "cool feature", it was something they had to be able to do.
I imagine that when storage became more plentiful OS developers became less interested in implementing solutions to problems of this nature and who can blame them really. Supporting a use-case that almost no one cares about anymore is rather unrewarding.
As others have mentioned, there are plenty of ways to achieve this on Linux today, they just seem to involve more fiddling than you'd like. That, unfortunately, is the nature of the beast.
Linux is a tinkerers OS (that just happened to become popular for servers) and expecting it to change its stripes is asking a lot :-)
You say this like making an application this way is some difficult problem. The whole point of me pointing out that it was the way applications worked on a Mac in 1984 (or indeed any of a dozen other desktop OSs) is that it isn't actually difficult at all. The difficulty it appears to have on Linux is all artificially induced by the culture of Linux software development.
> Linux is a tinkerers OS (that just happened to become popular for servers) and expecting it to change its stripes is asking a lot :-)
I don't expect it to change. I've been hoping Linux people would see the light for 20 years and it hasn't changed. That doesn't mean I have to like it or pretend that it's a good idea.
You probably know this already, but the point of Flatpak is to offer sandboxing and packaging of all dependencies in an isolated way. Being able to install and run flatpaks from any directory seems like it’d be difficult to implement for little benefit. It’s a bad fit for the tech IMO, and unnecessary since AppImage already offers exactly that functionality, but without the sandboxing (and other) features.