I really feel like easy internet access has taken good documentation away from us. It's too easy to Google for something, find some crappy blog with barely enough instructions to fix the problem. We've become reactive, rather than pro-actively educating ourselves.
I find the Gentoo wiki has the clearest information about specific things, and the Arch wiki has the clearest information on how to put them together.
This is "Arch is unstable" thing is very much a myth pushed forward by people who only dabbled with Arch once or twice then had it break, then completely dismissed it as unstable. Without first learning how to use it even at an intermediate level - not even expert.
I too had Arch break when I was a newbie but I learned quickly how not to and it's been incredibly stable ever since. Not worse than Debian and more so than Fedora in my experience.
In retrospect my actions which had broke it originally had little to do with ArchLinux itself but a general inexperience with Linux and over-eagerness when using beta software (which you have to opt in to, it's not default with Arch).
The end result is that I've come out of it a far better Linux user with skills highly applicable to Debian and other distros. It's a far easy place to learn Linux/Unix properly and become familiarized with the OS/dir structure/config than any other distro.
I don’t have to be careful about opting in to beta software or anything else with NixOS. If I want to try something, I can do so, and roll right back if it doesn’t work out. The destructive nature of Arch package installs and updates is its biggest flaw.
This accurately describes my experience. I stopped using Arch around the time git broke when I updated. I'm not clear what mistake I made. What level of Arch intermediate or expert level Arch knowledge is required to keep git working?
I've been running Arch for about three years now and I've had one very minor problem when I didn't read the notes properly.
The maintainers never seem to push anything not security related, most of the changes being backports of fixes from upstream.
Obliviously the price of stability is that the system runs slightly old software versions, but together with containers this is not a problem anymore. Stable OS running cutting edge containers is a combo that works well for me.
The policy for debian stable is to only push security fixes.
The corollary is that they don't push new version for bug fixes or features.
Example 1: Using System Python on Debian is a notoriously terrible experience for example, because they split up the venv module from Python itself and distribute it separately. I've had so many issues because of that and there's no good reason to do it. It's part of the stdlib, it doesn't need a separate package.
Example 2: Debian Jessie shipped with the ancient pip 1.5.6 (https://packages.debian.org/jessie/python-pip - upstream is at 9.0.1 since 2016) which never got updated, just stayed stuck there, wtf? Such old versions of pip are lacking support for a bunch of things that python packages today use on setup, so it would install things in a weird way and then packages would be broken, users wouldn't understand why. One of the many cases of not giving users access to a more recent version of a package, causing the user's experience to be severely impacted (and of course, they won't know to blame debian or the pip version in this case).
Edit: Incidentally, that last bit is why I always, always do `pip install --upgrade pip wheel setuptools` whenever I create a new virtual environment to make sure I don't get issues installing packages.
Having had quite a lot of experience of both, what I call, maintained packages and wild west packages (e.g. pypi), I would trust the former over the latter any day of the week. Developers generally do not make tremendously good maintainers. They tend to to only really care about the latest versions of anything, don't take particular care to make sure their package works well with other packages, rarely backport security fixes, and frequently do everything in a unilateral way. (I've seen it all, subtly different tarballs uploaded under the same version number, old releases deleted, breaking changes released under minor version bumps, addition of user-hostile "features", downloading & installing unexpected random binaries from the internet in attempt to make everything "just work", and of course they're forever bundling odd versions of other packages which they then don't maintain...)
Maintainers working on a "distribution" have a responsibility to make the whole distribution of software work well & stably together, and they generally do so quite well with the odd slip up (I'm looking in the direction of the debian-openssl screwup).
These days I tend to use a debian base system with Nix on top of it for when I need specific, repeatable or more recent versions of something.
In 2006, a big security flaw was introduced into Debian's OpenSSL by the maintainer, because he changed code that he did not fully understand. The flaw wasn't discovered until 2008:
https://www.schneier.com/blog/archives/2008/05/random_number...
But we've got to remember that attitudes to security and understanding of subtleties around crypto code were very different a decade ago, the output from automated checking tools was trusted significantly more, and we're talking about OpenSSL, a project whose source is generally acknowledged to be full of Weird And Inadvisable Shit anyway. The example you're leaning on was basically a perfect storm that resulted in the worst possible scenario.
On the other hand, package maintainers fix security issues on versions of packages the developers have stopped caring about every day of the week, and of course these never hit the headlines.
And of course I'd never suggest that any package maintainer could ever know better than you. Though unfortunately such superior and unilateral attitudes are one of the things that maintainers are needed to protect against.
Is there a policy that requires Debian package maintainers to seek mentorship of upstream when patching core packages?
I understand your point about maintained packages. But it's only a significant advantage if the maintainer's decisions must be filtered through the lens of the software maintainers. Otherwise one must trade the narrow expertise of the software maintainer for the wider expertise of the package maintainer.
That doesn't seem like a wise tradeoff IMO, and I don't see how a future openssl debacle could be prevented otherwise.
From a distribution perspective I sometimes wonder if it wouldn't be a better idea to call the system python/perl something like system-python/system-perl to make it more obvious they are not meant for use, and have a /usr/bin/python that tells you to install pyenv...
pip relies on bundled libraries and tries hard to fight any unbundling (by using slightly patched versions of the dependencies if I remember correctly). This makes it difficult to package. You can see as an additional proof of Debian messing with packages while on the Debian side, this is just the application of a distro-wide policy. If every piece of software should get an exception to the rule, we don't have a distribution anymore.
And from what i can tell, quite a bit of upstream would see that as a good thing. They just want to stuff their thing into a container and call it a day, distros be damned.
To me that is like going back to the DOS days of software, or even close to the present day of Windows software, where everything is dangling off in their own sub-dir with all their snowflake variants of dependencies.
Damn it, go back a few years and Microsoft go burned by this when they discovered that there was a vulnerability in the VC++ redistributable dlls. All they could do was issue updates for their own software, a tool for scanning the drive for vulnerable copies of the dlls, and a wish that customers would badger their software providers for updated versions of affected software.
Their policy on language specific package management is also pretty reasonable. There's no point duplicating effort packaging things in both Pip and the Debian repos when those package managers work fine under Debian. It lets Python devs use the same tools as every other Python developer on other systems, and avoids Debian specific instructions for package installation. It also completely side steps the problem of following upstream for all of those packages.
The advice I've always followed is to always use the language specific package management unless I'm specifically building a package for Debian.
The author of xscreensaver became so upset by similar circumstances that he threw down a gauntlet; ship the ancient package with an updated splash screen warning users of how old it is, or stop shipping it.
https://www.jwz.org/blog/2016/04/i-would-like-debian-to-stop...
Then again JWZ is quite the character...