And if you wanted your program propagated to all major distros you'd have to wait a decade.
Nobody has time for that. Not corporations, not mom and pop stores, and I doubt many hobbyists.
If you want to make your program popular then packaging is part of the process needed.
* Was "isn't that a problem?", which was not I meant to say.
That is demonstrably not true. I'm not making a value judgment (maybe it SHOULD be true, I dunno), but it's not true. There's a lot of popular unpackaged software out there.
I'm not sure I'm convinced based on the semi-frequent posts from maintainers and security pros about the issues with vendoring dependencies for software that is widely deployed.
This "better way", since we lack a more concrete name. Seems to be really great if you're running a web app, or server software in your own company and can rebuild and run a rolling deploy pretty easily. For someone pushing software to users all over the world, and as one of those users the downside to allow every application to be responsible for updating this stuff seems pretty steep.
> that's cutting maintainers out of the loop
Is this necessarily a bad thing? The market has seen the need to fill a hole, and it seems to be working.
I first started with Slackware, and dependency nightmares is what got me into Debian in the first place. Although Debian is nice because of its slow and stable base (which makes me happy for production), I’ve recently moved to Arch and have been so happy as it’s brought back Slackware’s idea of getting as close to upstream as possible and it handles dependencies! And to be honest, I’m loving it. And as an even added bonus, I’m getting more and more surprised how a tonne of packages that I’ve installed are Rust apps.
So, coming back to your comment:
> that's cutting maintainers out of the loop
With systems like Arch that get us closer and closer to upstream, are maintainers the unnecessary middlemen? Of course they’re not entirely redundant, but maybe a new model of distros like Arch will be more commonplace in the future
Arch is an old distribution, very much in the same class as Fedora, Debian, Gentoo and all the traditional ones.
What makes you think Arch makes maintainers even slightly redundant?
We still deal with security issues. We still need to figure out which Go software utilizes a library with a CVE with no tooling (go list and grep comes a long way). And we still need to deal with pinned dependencies in upstream projects.
AUR: Anyone can create an account and upload PKGBUILDs. There are no checks at all. AUR users should verify whether PKGBUILDs are not malicious. In practice, a lot of people use things like yaourt to install packages from the AUR without verifying the PKGBUILDs.
nixpkgs: anyone can contribute a PR with a new package, package update, or package modification. However, changes only get added to nixpkgs after someone with commit privileges verifies the PR and merges it. Also, a common misconception is that nixpkgs package maintainers can merge changes. This is false, only a much smaller set of committers can merge changes in the actual nixpkgs repository.
nixpkgs is more like the Arch Community repository, where committers are long-time contributors with a track record of high-quality contributions. Parts of nixpkgs are like Arch Core/Extra, because they are marked using the GitHub codeowners mechanism and changes are generally not merged unless approved through the code owners.
Disclaimer: I am a nixpkgs committer, former Arch user and AUR contributor.
The AUR comment is unfair, uncalled for and adds nothing to the conversation. I apologize and hope our previous conversations have been more productive. It was meant more tongue in cheek then some grand claim about the quality of nixpkgs and stems mostly from the frustration of the entire vendoring issue.
I don’t see your point. In fact, you’ll most likely find that distros that follow upstream closer than “slow and stable releases” will get their patches as soon as upstream fixes them
If someone publishes a CVE for a Go or Rust library it's not always the case the project is well maintained or the dev cares to update the dependency. Even if they did, there are no guarantees the upstream decides to publish a minor release just to update dependencies. Because that is what vendoring dependencies get you.
Instead of applying one patch to a shared library I'd need to hunt down all upstreams utilizing the library and manually patch between 10-140 packages independently and submit them upstream.
i.e. never. Lot of upstreams are abandoned with tons of PR's waiting years for somebody to merge in.
That said, I disagree that this means maintainers are unnecessary middlemen, even though their role on a distribution like Debian is obviously more prominent. The essay I linked to in my top level comment is actually by an Arch maintainer, explaining why they still see maintainers as playing an important role. http://kmkeen.com/maintainers-matter/
The userland is much more complex now, so freezing dependencies and bundling them in order to have a smaller number of test cases may not have been necessary when the environment was smaller and simpler. The reliability expectations were lower, and there weren't big money professional support contracts which required you to validate your application against a set of well defined environments. All that has changed.
And mind you, this is not a corporate/open source split. The burgeoning ecosystems are also full of FOSS technologies doing new and exciting things, they just don't break when a dependency updates!
>Anything I install with a package manager just works, 100% of the time
I run into issues with packages weekly. So much so I've spent engineer days purging references to system packages. It's universal too - yum, apt, pacman, brew, macports, I have to make sure nothing tries to reference packages installed outside a local working directory for an application because of mutual incompatibilities. Maybe it's because I'm trying to write software that runs on multiple targets and not use software where someone else has already spent the time and money to resolve these issues.
> I know you know this, but just to be clear, that's not a solution to the problem of "making things easier for maintainers to distribute", that's cutting maintainers out of the loop. The whole point of my focus on ecosystems is that this is something that I, as a user, don't want to happen.
They should be cut out of the loop. Maintainers don't have a right to dictate what design decisions I put into my applications because they don't think it adds value (the value is, it doesn't just run on their distro!). Another comment in this thread put it better, maintainers shouldn't place themselves in the development process.
I disagree. This is how you get the Google Play store or the "freeware" app marketplace. It sucks. As a user, I'm quite happy to continue using a traditional distribution even if it means I don't get to use a handful of flashy programs by developers that disagree with the concept of maintainers. So far that choice has been much more positive than negative for me, and I'm doing my best (by promoting the open source ecosystem) to keep it that way.
> Maintainers don't have a right
If you're creating open source software, they quite literally do! https://www.gnu.org/licenses/gpl-3.0.en.html
If you put in a bunch of crap that doesn't belong in an application (ads, for example), I'm glad I have a maintainer that can either strip this out thanks to the GPL (or BSD / MIT etc), or else choose not to include your app in the distribution at all.
As a user, I'm afraid that you don't seem to be a representative of typical users. I would be happy to use a traditional distribution if they don't break on updates, which is still not the case. It's clear that traditional distros were not satisfactory on even keeping their own promises. I know they work hard, but ultimately the visible outcome says everything.
This only works if "core" refuses to use Rust, so it's not sustainable. We already can't build Firefox and GNOME without Rust. Maybe Apache and curl next. And then?
Clearly you missed the disclaimer of warranty in the licensing terms.
Packagers are welcome to maintain their own patches, or their own fork, if they like. But they don't have any right to tell upstream what to do or demand particular guarantees from upstream.
Then perhaps we as developers should spend more time thinking about and participating in the maintenance process?
It seems like have posts every few months where security professionals, maintainers and sysadmins explain that the "developer usability first" approach to maintenance and dependency management is having massive consequences to our ability to keep systems secure, and to even know if an affected version of a library is on a system.
That is a different problem that doesn't seem to be addressed at all in the current iteration of these tools. If you're aware of efforts to solve that problem in the rust and go ecosystems(since those are the ones cited here) I'd love to read about it.
Frankly, most distribution maintainers seem to not know or care about how upstream software is built; they have an idea about what's "best practice" in the handful of languages they're using to build their distribution (which is mostly, like, C and Perl) and insist that they know best, without realising the rest of the world has passed them by.
Well, that's exactly why the OP (and my essay) are "anti" modern software development in many ways. The view is that we're moving away from the traditional open source ecosystem and methods of software development with these new technologies, which (to be clear) are good technologies, but were created mostly to solve problems that some large corporations have, not to solve the problems that the open source ecosystem has.
> The least reliable way to package and distribute software is by relying on traditional package managers.
Not sure what you mean by this, but it's entirely untrue in my experience. Anything I install with a package manager just works, 100% of the time. Stuff I try to get any other way is a shitshow, and the lack of "quality control" provided by maintainers speaks for itself. I mean, just look at Android apps or the Chrome extension store. Heaven forbid we go back to the days of curl | bash off someone's website.
> By statically linking everything as much as possible and shipping everything else in a self contained bundle with a launcher that overrides any symbols that might inadvertently be pulled in from the system.
I know you know this, but just to be clear, that's not a solution to the problem of "making things easier for maintainers to distribute", that's cutting maintainers out of the loop. The whole point of my focus on ecosystems is that this is something that I, as a user, don't want to happen.