News for hackers. Stuff that matters.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by tmoravec on Wednesday December 16, 2020 @01:45PM (#25446573)
Debian offers around three years of support. Ubuntu LTS around five. Both pale in comparison with Red Hat and, by proxy, CentOS.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by geofft on Wednesday December 16, 2020 @03:08PM (#25447846)
Debian eLTS offers 7 years: https://wiki.debian.org/LTS/Extended

That said, if you're really in the position of depending on a free project for over five years of security support, you probably will be totally fine with just ignoring the fact it's out of support. Just keep running Debian 6 for a decade, whatever. The code still runs. Pretend you've patched. Sure, there are probably some vulnerabilities, but you haven't actually looked to see if the project you're actually using right now has patched all the known vulnerabilities, have you?

(Spoiler, it hasn't: https://arxiv.org/abs/0904.4058)

Re:Rocky Linux: A CentOS replacement by the CentOS founder by joana035 on Wednesday December 16, 2020 @01:48PM (#25446622)
Right, but the price one pays are outdated packages.

CentOS 8 was released few days ago with kernel 4.18, which not even LTS is, and is older than the current Debian stable kernel(!).

If you need to install anything besides the base distro you need elrepo, epel, etc which I'm not sure can be counted as part of the support.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by dralley on Wednesday December 16, 2020 @02:43PM (#25447484)
RHEL kernel versions are basically incomparable with vanilla kernel versions. They have hardware support and occasionally entire new features that have been backported from newer kernels in addition to the standard security & stability patches.

This means that RHEL 7 using a "kernel version" from 2014 will still work fine with modern hardware for which drivers didn't even exist in 2014.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by the8472 on Wednesday December 16, 2020 @03:01PM (#25447752)
That is not a good thing. RH frankenkernels can contain subtle breakage. E.g. the Go and Rust standard libraries needed to add workarounds because certain RH versions implemented copy_file_range in a manner that returns error codes inconsistent with the documented API because patches were only backported for some filesystems but not for others. These issues never occurred on mainline.

And for the same reasons that the affected users chose a "stable" and "supported" distro they were also unable to upgrade to one where the issue was fixed.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by dralley on Wednesday December 16, 2020 @03:22PM (#25448055)
True, but it is a matter of weighing risks. I can't find it now, but I remember a few years ago there was a news story about how an update to Ubuntu had caused hospitals to start rendering MRI scan results differently due to differences in the OpenGL libraries. For those sorts of use cases, stable is the only option.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by smarx007 on Wednesday December 16, 2020 @05:45PM (#25449816)
I think this is a perfect use case for CentOS/RHEL as opposed to Ubuntu when the machine has only one job and nothing shall stand in its way, ie when you expect everything to be bug-for-bug compatible. But I fail to understand why a vendor of an MRI machine charging tens of thousands for installation/support cannot provide a supported RHEL OS which costs $180-350/yr in the cheapest config [1].

[1]: https://www.redhat.com/en/store/linux-platforms

Re:Rocky Linux: A CentOS replacement by the CentOS founder by pezezin on Thursday December 17, 2020 @09:17AM (#25455735)
No, it doesn't. I will tell you my experience.

They bought some fancy new computers at work. Our procedures say to use CentOS 7, so we tried it, it ran like shit. Then we reinstalled CentOS 8, same. It worked, but the desktop was extremely slow. After much hair pulling I found the solution: add the elrepo-kernel repository, and update to kernel 5.x

No amount of backporting magic will make an old kernel work like a new kernel.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by naniwaduni on Wednesday December 16, 2020 @02:15PM (#25447087)
That's not a price, that's a feature.

If your use case doesn't consider avoiding noncritical behaviour changes for a decade to be a feature, you have other options.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by syshum on Wednesday December 16, 2020 @02:41PM (#25447458)
if you need epel, or quicker life cycles then CentOS Stream should be just fine for you as well

People that run CentOS in prod are normally running ERP systems, Databases, LoB Apps, etc, and the only thing we need is the base distro and the the vendor binaries for what ever is service/app that needs to be installed, and probably an old ass version is JDK...

We need every bit of that 10 year life cycle, and we glad that we will probably only have to rebuild these systems 2 or 3 times in our career before we pass the torch to the next unlucky SOB that has to support an application that was written before we were born...

that is CentOS Administration ;)

Re:Rocky Linux: A CentOS replacement by the CentOS founder by CoolGuySteve on Wednesday December 16, 2020 @02:07PM (#25446936)
The patch delta for security fixes must get larger over time as these packages age further and further away from top of tree.

I always wonder how many major vulnerabilities are introduced into these super old distros due to backporting bugs.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by bonzini on Wednesday December 16, 2020 @03:26PM (#25448116)
It's the opposite. Plenty of subsystems in the RHEL 8.3 kernel are basically on par with upstream 5.5 or so, as almost all the patches are backported. The source code is really the same to a large extent, and therefore security fixes apply straightforwardly.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by Tom4hawk on Thursday December 17, 2020 @05:49AM (#25454407)
So, why is RHEL not using the upstream kernel? It would allow them to avoid those issues with rust&go (and probably other software): https://news.ycombinator.com/item?id=25447752
Re:Rocky Linux: A CentOS replacement by the CentOS founder by bonzini on Thursday December 17, 2020 @07:44AM (#25455047)
RHEL maintains a stable ABI for drivers.

Plus, there are changes (especially around memory management or scheduling) that are fiendishly hard to do regression testing on, so they are backported more selectively.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by waheoo on Thursday December 17, 2020 @07:14AM (#25454881)
Security audit / certification would be my guess.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by CoolGuySteve on Wednesday December 16, 2020 @05:19PM (#25449567)
That's great but what about all the other packages?
Re:Rocky Linux: A CentOS replacement by the CentOS founder by bonzini on Thursday December 17, 2020 @02:44AM (#25453520)
The upstream for most other packages generally move much more slowly than the kernel. The fast ones (e.g. X11, systemd, QEMU) are typically rebased every other update or so (meaning, roughly once a year).

It also helps that Red Hat employs a lot of core developers for those fast moving packages. :)

Re:Rocky Linux: A CentOS replacement by the CentOS founder by jtl999 on Wednesday December 16, 2020 @02:11PM (#25447016)
Documented cases don't seem to be common, but what comes to mind is the Debian "weak keys" scandal (2008), and the VLC "libeml" vulnerability (2019)[1]

[1]: https://old.reddit.com/r/netsec/comments/ch86o6/vlc_security...

Re:Rocky Linux: A CentOS replacement by the CentOS founder by joana035 on Wednesday December 16, 2020 @02:19PM (#25447140)
OpenSSL upstream was almost abandoned during those days.

Software are always gonna have bugs, it's written by humans after all. The important thing is to acknowledge and work towards an ideal outcome.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by kasabali on Thursday December 17, 2020 @08:49AM (#25455512)
Xweak keys" didn't have anything to do with backporting fixes to older versions. It was introduced into the version in sid at the time.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by houseofzeus on Wednesday December 16, 2020 @04:03PM (#25448619)
The kind of people that are up in arms that CentOS 8 isn't going to be supported through to 2029 are using it because it has outdated packages.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by citiguy on Wednesday December 16, 2020 @01:51PM (#25446686)
Agreed, the packages in Centos / RHEL are all super old. The RHEL license structure changes all the time and depending on which one you get it may or may not include the extended repos.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by mmrezaie on Wednesday December 16, 2020 @02:00PM (#25446832)
Honestly that support is meaningless for some areas I know. In our Data Center we have hit problems with old packages and at the end you will end up with a lot of your own packages. In the end I find Debian to be a good base, and you build the rest by yourself. Even though I use Fedora for Desktop, I always have a feeling Debian is the server choice which I can extend further.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by goodpoint on Wednesday December 16, 2020 @03:37PM (#25448252)
This is false. Debian provides LTS with a 5-years timespan. [1]

And there is even commercial support for Extended LTS now [2]

Also, it's worth noticing that Debian provides security backports for a significantly larger set of packages and CPU architectures than other distributions.

[1] https://wiki.debian.org/DebianReleases

[2] https://wiki.debian.org/LTS/Extended

Re:Rocky Linux: A CentOS replacement by the CentOS founder by antoncohen on Wednesday December 16, 2020 @04:36PM (#25449084)
Do you trust Debian LTS? As much as RHEL? The documentation about Debian LTS always made me think it is not a fully fledged thing. I've always felt like Debian releases reached EOL on their EOL date, not their LTS EOL date.

> Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success.

https://wiki.debian.org/LTS

Re:Rocky Linux: A CentOS replacement by the CentOS founder by tmoravec on Wednesday December 16, 2020 @03:49PM (#25448424)
Good catch, I stand corrected. But the point still holds - five (or even seven) years is still not match for Red Hat.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by uep on Wednesday December 16, 2020 @05:11PM (#25449480)
Do you know something I don't? A few years back, Debian changed their LTS policy to 5 years in response to Ubuntu.

> Debian Long Term Support (LTS) is a project to extend the lifetime of all Debian stable releases to (at least) 5 years. Debian LTS is not handled by the Debian security team, but by a separate group of volunteers and companies interested in making it a success.

> Thus the Debian LTS team takes over security maintenance of the various releases once the Debian Security team stops its work.

https://wiki.debian.org/LTS

Re:Rocky Linux: A CentOS replacement by the CentOS founder by unethical_ban on Wednesday December 16, 2020 @02:05PM (#25446915)
Arguably, no one should be running a server that long in 2020.

I would say a better reason is that while both are Linux distributions, they are distinct dialects and ecosystems. It isn't impossible to switch, but for institutions that have complex infrastructure built around the RHEL world, it is a lot of work to convert.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by chousuke on Wednesday December 16, 2020 @02:50PM (#25447598)
It's not really about running servers for 10 years. It's about having a platform to build a product on that you can support for 10 years. RHEL software gets old over time, but it's still maintained and compatible with what you started on.

Consider an appliance that will be shipped to a literal cave for some mining operation. Do you want to build that on something that you would have to keep refreshing every year, so that every appliance you ship ends up running on a different foundation?

Re:Rocky Linux: A CentOS replacement by the CentOS founder by brandonmenc on Wednesday December 16, 2020 @03:03PM (#25447785)
> Consider an appliance that will be shipped to a literal cave

This.

A decade ago I was technical co-founder of a company [0] that made interactive photo booths and I chose CentOS for the OS.

There are some out in the wild still working and powered on 24/7 and not a peep from any of them.

We only ever did a few manual updates early on - after determining that the spotty, expensive cellular wasn't worth wasting on non-security updates - so most of them are running whatever version was out ten years ago.

Rock solid.

[0] https://sooh.com

Re:Rocky Linux: A CentOS replacement by the CentOS founder by deckard1 on Wednesday December 16, 2020 @03:35PM (#25448234)
The "don't touch it if it's not broken" philosophy is fundamentally at odds with an internet-connected machine.

You either need to upgrade or unplug (from the internet).

There are still places out there that are running WindowsNT or DOS even. Because they have applications which simply won't run anywhere else or need to talk to ancient hardware that runs over a parallel port or some weird crap like that. These machines will literally run forever, but you wouldn't connect it to the internet. Your hypothetical cave device would be the same.

Upgrading your OS always carries risk. Whether it's a single yum command or copying your entire app to a new OS.

Besides, if you're on CentOS 8 then wouldn't you also be looking at Docker or something? Isn't this a solved problem?

Re:Rocky Linux: A CentOS replacement by the CentOS founder by chousuke on Thursday December 17, 2020 @03:15AM (#25453672)
The point is the amount of "touching". Applying security patches to RHEL is still a change, but it's significantly less risky than upgrading a faster-changing system where you might not even get security patches at all for the versions of software you're using unless you switch to a newer major version.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by effie on Wednesday December 16, 2020 @05:00PM (#25449378)
"don't touch it if it's not broken" is not a philosophy, it is a slogan. Some people say it, because it is preferable to them to run old unpatched vulnerable systems rather than spend resources on upgrades. That's just a reality. Some care about up-to-date, some don't. Most people don't really care about security, and some of those don't care even about CYA security theatre. If they did care about security, they wouldn't run unverified software downloaded from the Internet.

Why Docker has anything to do with this discussion?

Re:Rocky Linux: A CentOS replacement by the CentOS founder by effie on Wednesday December 16, 2020 @05:05PM (#25449424)
I think it is mostly about running servers (standard services that don't change much) for 10 years (and more). You don't need 10 year LTS distribution for building a product. You take whatever version of OS distribution you like, secure local copy should the upstream disappear, and vendor it into your product and never deviate from it.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by dmix on Wednesday December 16, 2020 @03:02PM (#25447774)
I'm assuming this involve upgrading the application often as well?
Re:Rocky Linux: A CentOS replacement by the CentOS founder by NotHereNotThere on Wednesday December 16, 2020 @02:59PM (#25447720)
OK I'll bite..

How would I run our SAP ERP apps/databases without "running servers that long"?

Re:Rocky Linux: A CentOS replacement by the CentOS founder by unethical_ban on Wednesday December 16, 2020 @03:38PM (#25448261)
Of course there are use cases, but _ideally_, most workloads are staged, deployed, and backed up in such a way that it is a documented, reproducible procedure to tear down an instance of a server, rebuild, and redeploy services.

And while it may be cumbersome or cause some downtime or headaches if that isn't the case, I find the very need of doing it once every 1-3 years forces your hand to get your shit together, rather than a once per decade affair of praying you migrated all your scripts manually and that everything will work, as your OS admins threaten your life because audit is threatening theirs.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by effie on Wednesday December 16, 2020 @05:09PM (#25449460)
How many simultaneously running machines can you keep updating with this method? If you run non-trivial workloads for hundreds of customers, this becomes high maintenance system already with two machines. It takes ages to upgrade all applications, then validate everything works, then actually migrate with no downtime.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by petepete on Wednesday December 16, 2020 @03:23PM (#25448065)
Run them on Kubernetes in GCP with a containerised DB2!
Re:Rocky Linux: A CentOS replacement by the CentOS founder by joana035 on Wednesday December 16, 2020 @02:12PM (#25447027)
> Arguably, no one should be running a server that long in 2020

The main reason people choose Linux is due to its stability.

It is totally ok to run servers in 2020.

Also your statement seems the default cloud vendor lingo used to push people to adopt proprietary technology with high vendor lock-in.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by mdeeks on Wednesday December 16, 2020 @02:54PM (#25447663)
I think they only meant "that long". In other words, not for 5+ years without an upgrade. Not that you shouldn't run them at all.

Not everything can be highly ephemeral or a managed service, so running servers yourself is totally okay like you said.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by joana035 on Wednesday December 16, 2020 @03:24PM (#25448073)
You are right, I misunderstood his message.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by that_guy_iain on Wednesday December 16, 2020 @02:54PM (#25447669)
Honestly, 10 years is a long time for a server. I would be honestly surprised if a server lasted 10 years.

But I agree I also get the tone of "servers should be cattle and not pets, just kill them and build a new one". Which can also be done on bare metal if you're using vms/containers. It seems like most people forget these cloud servers need to run on bare metal.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by bcrosby95 on Wednesday December 16, 2020 @03:29PM (#25448152)
Really? We've colocated our servers for the past 18 or so years.

We have about 40. The oldest is around 17 years old. Our newest server is 9 years old. Our average server age is probably around 13 years old.

The most common failure that completely takes them out of commission is a popped capacitor on the motherboard. Never had it happen before the 10 year mark.

Never had memory failure. Have had disk failures, but those are easy to replace. Had one power supply failure, but it was a faulty batch and happened within 2 years of the server's life.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by that_guy_iain on Wednesday December 16, 2020 @04:29PM (#25448977)
The last time I worked with a ~8 year old server, it used to go through hard drives at a rate of 1 every 2 months. While we could replace them easily and it was RAID so there wasn't any data loss, I personally would've got fed up of replacing HDDs every couple of months.

Also, most of my experience is with rented dedicated servers and they just give me a new one completely so I never really see if they're fully scrapped.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by bcrosby95 on Wednesday December 16, 2020 @07:05PM (#25450524)
We haven't had any hardware failures - hard drive or otherwise - in the last 5 months.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by moonbug on Wednesday December 16, 2020 @02:32PM (#25447355)
>Arguably, no one should be running a server that long in 2020.

That's mental.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by Karunamon on Wednesday December 16, 2020 @02:39PM (#25447431)
My read on that is that you should be treating your servers as disposable and ephemeral as possible. Long uptimes mean configuration drift, general snowflakery, difficulties patching, patches getting delayed/not done, and so forth.

Ideally you'd never upgrade your software in the usual way. You'd simply deploy the new version with automated tooling and tear down the older.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by hibbelig on Wednesday December 16, 2020 @03:32PM (#25448191)
I don't get this. If there are many servers, sure. But if it's something that runs on a single box without problem, why on earth should I tear it down?

Also, "running a server for ten years" does not need to mean that it has ten years of uptime. I think that wasn't meant.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by moonbug on Wednesday December 16, 2020 @04:54PM (#25449305)
Ten years of uptime seems neither an unreasonable nor unattainable requirement. There's more to computers than mayfly web startups.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by hibbelig on Thursday December 17, 2020 @11:34AM (#25457125)
If it's not connected to the Internet, okay.

If it is connected to the Internet, then I guess the kernel needs to be hot-patched need to be applied to avoid security issues.

Were hot kernel patches available ten years ago? I remember some company who did this (for Linux), and it was quite a while back, so it's possible. But I doubt it was mainstream.

I recall long ago that SunOS boxes had to be rebooted for kernel patches.

I don't remember about Solaris.

I'm not familiar with other Unices.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by moonbug on Thursday December 24, 2020 @08:20AM (#25527190)
ksplice, and it's an Oracle product
Re:Rocky Linux: A CentOS replacement by the CentOS founder by NDizzle on Wednesday December 16, 2020 @02:51PM (#25447605)
"Ideally" - that's the problem. I have half a dozen long tail side projects running right now on Centos 7, and a few still on Centos 6.

Do you have any idea how much effort it is to change everything over to "treating your servers as disposable"?! It's going to eat up a third (to half) of my "fun time" budget for the foreseeable future!

Re:Rocky Linux: A CentOS replacement by the CentOS founder by effie on Wednesday December 16, 2020 @05:18PM (#25449559)
Exactly, young devs here are completely out of touch with operations. Of course ideally something like standard 1TB HDD+32GB RAM system would be upgraded to newer OS and apps version by a central tool in 2hours, but we don't such FM technology yet.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by cat199 on Wednesday December 16, 2020 @03:45PM (#25448354)
> automated tooling

the 'usual way' is automated tooling