News for hackers. Stuff that matters.
Re:Debian discusses vendoring again by initplus on Wednesday January 13, 2021 @03:20AM (#25759528)
I am worried here that the alternative we'll end up with is applications that rely on vendoring will end up distributed entirely outside the Debian repositories... hopefully with go get/npm install, hopefully not with "download it from my website!"... But either way you lose a lot of the benefits that being officially in the Debian repos would bring. Devs want to distribute their software to users, and they aren't going to chase down rabbit holes to get it packaged to comply with every different distributions set of available dependency versions.

Really this idea that a distro (even a large well maintained one like Debian) has the resources to package a set of known versions of go/node packages for common open source software seems wrong? If they aren't going to package every exact version that's required, how is it going to be possible to test for compatibility? There is no way. And no dev is going to downgrade some random dependency of their app just to comply with with Debian's set of available versions.

Developers hate this versioning issue with languages like C\C++ on Linux, it's a huge pain. And that's partially why dependency management in languages like Go/Node work the way they do. A multitude of distros with slightly different versions of every lib you use is a huge headache to dev for, so people have designed languages to avoid that issue.

Re:Debian discusses vendoring again by throwaway098237 on Wednesday January 13, 2021 @11:24AM (#25764148)
There' has been always a split between software that is expected to be run for 10 or 20 years and software that will be obsoleted in 2 years.

https://www.cip-project.org/ aims to backport fixes to released kernel for 25 (twenty-five) years.

Because you don't "npm update" deployed systems on: banks, power plants, airplanes and airports, trains, industrial automation, phone stations, satellites. Not to mention military stuff.

(And Debian is much more popular in those places that people believe)

> Devs want to distribute their software to users, and they aren't going to chase down rabbit holes to get it packaged to comply with every different distributions set of available dependency versions.

That's what stable ABIs are for.

> Really this idea that a distro (even a large well maintained one like Debian) has the resources to package a set of known versions of go/node packages for common open source software seems wrong?

Yes, incredibly so. Picking after lazy developers to unbundle a library can require hours.

Backporting security fixes for hundreds of thousands libraries including multiple versions is practically impossible.

> And no dev is going to downgrade some random dependency of their app just to comply with with Debian's set of available versions.

Important systems will keep running in the next decades. Without the work from such developers.

Re:Debian discusses vendoring again by jillesvangurp on Wednesday January 13, 2021 @04:24AM (#25759904)
That's already the reality for most of this century. Openjdk, go, rust, docker, npm/yarn, etc. all provide up to date Debian, Red Hat, etc. packages for what they offer. There's zero advantage to sticking with the distribution specific versions of those packages which are typically out of date and come with distribution specific issues (including stability and security issues).

Debian's claims to adding value in terms of security and stability to those vendor provided packages are IMHO dubious at best. At best they sort of ship security patches with significant delays by trying to keep up with their stable release channels. Worst case they botch the job, ship them years late, or introduce new bugs repackaging the software (I experienced all of that at some point).

When it comes to supporting outdated versions of e.g. JDKs, there are several companies specializing in that that actually work with Oracle to provide patched, tested, and certified JDKs (e.g. Amazon Coretto, Azul, or Adopt OpenJDK). Of course for Java, licensing the test suite is also a thing. Debian is probably not a licensee given the weird restrictive licensing for that. Which implies their packages don't actually receive the same level of testing as before mentioned ways of getting a supported JDK.

On development machines, I tend to use things like pyenv, jenv, sdkman, nvm, etc. to create project specific installations. Installing any project specific stuff globally is just unprofessional at this point and completely unnecessary. Also, aligning the same versions of runtimes, libraries, tools, etc. with your colleagues using mac, windows, and misc. Linux distributions is probably a good thing. Especially when that also lines up with what you are using in production.

Such development tools of course have no reason to exist on a production server. Which is why docker is so nice since you pre-package exactly what you need at build time rather than just in time installing run-time dependencies at deploy time and hoping that will still work the same way five years later. Clean separation of infrastructure deployment and software deployment and understanding that these are two things that happen at separate points in time is core to this. Debian package management is not appropriate for the latter.

Shipping tested, fully integrated, self-contained binary images is the best way to ship software to production these days. You sidestep distribution specific packaging issues entirely that way and all of the subtle issues that happen when these distributions are updated. If you still want Debian package management, you can use it in docker form of course.

Re:Debian discusses vendoring again by Blikkentrekker on Wednesday January 13, 2021 @06:50AM (#25760916)
> That's already the reality for most of this century. Openjdk, go, rust, docker, npm/yarn, etc. all provide up to date Debian, Red Hat, etc. packages for what they offer. There's zero advantage to sticking with the distribution specific versions of those packages which are typically out of date and come with distribution specific issues (including stability and security issues).

The advantage is the very reason one would choose Debian to begin with — an inert, unchanging, documented system.

A large part of this problem seems to be that users somehow install a system such as Debian whose raison d'être is inertia, only to then complain about the inertia, which makes one wonder why they chose this system to begin with.

> Debian's claims to adding value in terms of security and stability to those vendor provided packages are IMHO dubious at best. At best they sort of ship security patches with significant delays by trying to keep up with their stable release channels. Worst case they botch the job, ship them years late, or introduce new bugs repackaging the software (I experienced all of that at some point).

Evidently they add value in terms of stability, but methinks many a man misunderstands what “stable" means in Debian's parlance. It does not mean “does not crash”; it means “is inert, unchanging” which is important for enterprises that absolutely cannot risk that something stop working on an upgrade.

> Shipping tested, fully integrated, self-contained binary images is the best way to ship software to production these days. You sidestep distribution specific packaging issues entirely that way and all of the subtle issues that happen when these distributions are updated. If you still want Debian package management, you can use it in docker form of course.

Not for the use case that Debian, and RHEL attempt to serve at all — these are systems that for good reasons do not fix non-critical bugs but rather document their behavior and rule them features, for someone might have come to rely upon the faulty behavior, and fixing it would lead to breaking such reliance.

Re:Debian discusses vendoring again by jillesvangurp on Wednesday January 13, 2021 @10:35AM (#25763248)
That's why most shops deploy docker containters: it's not convenient at all for them for Debian, Red Hat, etc. to repackage the software they deploy or be opinionated about what versions of stuff is supported. For such users, the OS is just a runtime and it just needs to get out of the way.

Ten years ago, we were all doing puppet, chef and what not to customize our deployment infrastructure to run our software. That's not a common thing anymore for a lot of teams and I have not had to do stuff like that for quite some time. A lot of that work btw. involved working around packaging issues and distribution specific or distribution version specific issues.

I remember looking at the puppet package for installing ntp once and being horrified at the hundred lines of code needed to run something like that because of all the differences between platforms. Also such simple things like going from one centos version to the next was a non trivial issue because of all the automation dependencies on stuff that changed in some way (I remember doing the v5 to v6 at some point). Dealing with madness like that is a PITA I don't miss at all.

There's definitely some value in having something that is inert and unchanging for some companies that run software for longer times. Pretty much all the solutions I mentioned have LTS channels. E.g. If you want java 6 or 7 support, you can still get that. And practically speaking, when that support runs out I don't see how Debian would be in any way positioned to provide that in a meaningful way. The type of company caring about such things would likely not be running Debian but some version of Red Hat or something similarly conservative.

Re:Debian discusses vendoring again by curt15 on Wednesday January 13, 2021 @09:44AM (#25762542)
>It does not mean “does not crash”; it means “is inert, unchanging” which is important for enterprises that absolutely cannot risk that something stop working on an upgrade.

But would enterprises accept being forever stuck with any bugs that aren't security related? Even RHEL backports patches from newer kernels while maintaining kABI.

Re:Debian discusses vendoring again by Blikkentrekker on Wednesday January 13, 2021 @11:06AM (#25763843)
We're talking about entitites that run COBOL code from the 60s and are too afraid to update or replace it, for fear that something break.

There's a reason why most enterprise-oriented systems take inertia quite seriously — it has been something their customers greatly desire who are losing considerable capital on even minor downtime.

Re:Debian discusses vendoring again by cbmuser on Wednesday January 13, 2021 @07:06AM (#25761025)
> Debian's claims to adding value in terms of security and stability to those vendor provided packages are IMHO dubious at best.

That’s not true. The idea is that the distribution is tested and stable as a whole and replacing something as OpenJDK can cause a lot breakage in other packages.

There is a reason why enterprise distributions provide support only for the limited set of packages that they ship.

Re:Debian discusses vendoring again by jillesvangurp on Wednesday January 13, 2021 @10:46AM (#25763444)
Depends, if you install a statically linked version from a third party it won't create much headaches. That kind of is the point of vendoring and static linking: not making to much assumptions about what is there and what version it is in. Works great at the cost of a few extra MB, which in most cases is a complete non issue for the user.

Debian self-inflicts this breakage by trying to share libraries and dependencies between packages. That both locks you in to obsolete stuff and creates inflexibility. Third parties actively try to not have this problem. Debian is more flaky on this front than it technically needs to be.

Kind of the point of the article is that to vendor or not to vendor is a hot topic for Debian exactly because of this.

Re:Debian discusses vendoring again by saagarjha on Wednesday January 13, 2021 @05:34PM (#25768918)
> There's zero advantage to sticking with the distribution specific versions of those packages which are typically out of date and come with distribution specific issues (including stability and security issues).

Uh, other than "apt install foo" versus "ok, let's go search for foo on the internet, skip that spam listing that Google sold ad space to, ok no I am on foo.net, let's find the one that corresponds to my computer…yeah amd64 Linux, rpms? no, I want debs…download, dpkg -i…oh wait I need libbar".

Re:Debian discusses vendoring again by npsimons on Wednesday January 13, 2021 @01:14PM (#25765877)
> But either way you lose a lot of the benefits that being officially in the Debian repos would bring.

The first thing I do when I hear about a new (to me) piece of software is an "apt-cache search $SOFTWARE". If it doesn't show up there, that's a red flag to me: this software isn't mature or stable enough to be trusted on my production machines.

Sure, I might go ahead and download it to play around with on my development machines, but for all the "I'm making it awesome!" arguments of developers, more often than not it's just an excuse for lack of discipline in development process.

Re:Debian discusses vendoring again by JoshTriplett on Wednesday January 13, 2021 @03:34AM (#25759613)
This was exactly my concern. I believe that Debian packages should avoid vendoring when possible, but that means it must be possible to package the individual modules, even if there are multiple versions, and even if there are many small dependencies.
Re:Debian discusses vendoring again by sorisos on Wednesday January 13, 2021 @04:36AM (#25759981)
Agree no one is going to downgrade but there is another strategy - always build your app against package versions that is in Debian stable. Of-course it can be problematic but have some advantages: well tested, any bugs probably have documented workaround.
Re:Debian discusses vendoring again by juststeve on Wednesday January 13, 2021 @11:11PM (#25771880)
Yep that was my take on it as well. Vendors should be able to test their software against a vanilla install of debian stable and build the deb package themselves, and then upload to the package repository for review. Else, the vendor provides instructions/support on their external website.