News for hackers. Stuff that matters.
Re:CentOS Project shifts focus to CentOS Stream by Jimmc414 on Tuesday December 08, 2020 @02:15PM (#25349989)
>> Imagine if you were running a business, and deployed CentOS 8 based on the 10 year lifespan promise. You're totally screwed now, and Red Hat knows it.

The hypothetical you posed is the actual situation, I am now learning, I have apparently forced on my team. We've ramped up labor 3x revenue preparing product launch in 90 - 180 days. We created an image containing centos 8 , Java , postgres and tomcat a year ago and that what is deployed to beta clients and what we've been testing.

What's ironic is that I sort of went out on a limb with my team by forcing us to go with Linux over Windows and the way I allayed concerns was to ask them to just "wait and see" in hopes that the performance differential would make it a moot point.

edit: after a little thought it seems that moving to RHEL might cost us the least amount of money and downtime. Still sucks and not what we need to be working on right now.

Re:CentOS Project shifts focus to CentOS Stream by kseifried on Tuesday December 08, 2020 @11:47PM (#25355472)
What I don't get is this:

You build software, you ship it on an OS so it works, cool, this makes sense (I assume you need hardware and VM support, not just VM).

Why would you accept additional risk on the OS if you can easily reduce the risk, and ultimate cost, by going with an OS that has vendor support written into the actual contract? RHEL is 11-13 years total, Windows is... I'm guessing my grandkids will be using some form of Windows 10. CentOS is, and always was a community "best effort", with some serious delays occasionally (not often, but it happened).

A RHEL server license starts at $349, I have to assume that's at least an order of magnitude (or two or three) less than the cost of your software based on the technologies involved (sounds enterprise-solutiony). In other words a rounding error overall.

Re:CentOS Project shifts focus to CentOS Stream by hjb100 on Thursday December 10, 2020 @03:50AM (#25371373)
As a sysadmin in academia, this is not so straightforward. Since number of servers/VMs are in ballpark of over one hundred, RedHat license costs will be over 30,000$/year. This is not insignificant amount of money and not easy to get the money suddenly.
Re:CentOS Project shifts focus to CentOS Stream by bierjunge on Thursday December 10, 2020 @07:14AM (#25372850)
$30k is a significant amount, especially in academic environment, I don't deny it. But it's not much for > 100 servers. In my previous job, I had licenses which were about 15k€ for one server (thanks Oracle...) and the Windows Server licenses were also pretty expensive. $300-500 per server/year is not much more than a Photoshop license for one designer. And if we talk big business (1000's of servers and even more), you will get very special pricing anyway.
Re:CentOS Project shifts focus to CentOS Stream by russfink on Thursday December 10, 2020 @10:30AM (#25374711)
Not just price, but ability to function without constant access to the satellite portal. It's darn near impossible to upgrade a RHEL box from a work at home VPN environment.
Re:CentOS Project shifts focus to CentOS Stream by fweespeech on Tuesday December 08, 2020 @04:00PM (#25351282)
> edit: after a little thought it seems that moving to RHEL might cost us the least amount of money and downtime. Still sucks and not what we need to be working on right now.

And that is exactly what IBM is counting on. Vendor lock in.

We are switching to Debian-based distros because, frankly, we don't trust IBM not to knife us in the back even on RHEL for more money. Of course we have the advantage of being able to take the time to convert.

Re:CentOS Project shifts focus to CentOS Stream by bonzini on Tuesday December 08, 2020 @04:00PM (#25351295)
This is not about IBM. It may not be a pleasant change for everyone, but it's a change that has strictly technical motives.

CentOS was acqui-hired because Red Hat's upstream for layered products (at the time mostly RDO/OpenStack and oVirt/RHEV) could not use Fedora because it was too far from RHEL a year of two after RHEL was released, could not use RHEL because upstream contributors would have to pay, and could not use CentOS because its releases had too large delays. The solution was to make CentOS releases happen timely by paying people to make them.

These days a RHEL downstream is not enough for the layered products. Some of them require the kind of bleeding edge feature that is backported every six months to the RHEL kernel, and corresponding userspace changes (BPF, virtualization, etc.) and cannot afford waiting for the CentOS release because development must be done in parallel with RHEL. So the solution was to move CentOS from happening after RHEL to before RHEL which is what CentOS Stream is.

I can confidently say that the reasons are technical because other CentOS downstream have the same needs (e.g. Facebook's) and they also want to send patches to CentOS for bugfixes or features themselves, instead of waiting for Red Hat to find out about that bug, or decide they need the same feature. Plus there's no reason for rebuilds to disappear. The SRPMs will still be released by Red Hat.

Re:CentOS Project shifts focus to CentOS Stream by kbenson on Tuesday December 08, 2020 @04:10PM (#25351392)
That in no way explains why they don't continue to have both. They indeed will have both for another year. There was no requirement the Stream product even use the CentOS name.

CentOS was a community project whose leadership and control was taken over (acqui-hired as you say) by Red Hat and then it's core use case for the majority of people actually using it was discontinued. That is a statement of facts that happened as I understand them, not some spin on my part.

If Red Hat had not stepped in, perhaps some of CentOS problems (trouble getting releases out on time) would have been worse, or perhaps some other companies would have stepped in. We don't know, but we do know that CentOS has not been changed to be something different than it was before. It used to be a free re-spin of RHEL. Going forward it's something entirely different.

Red Hat always had the option to stop funding/providing resources to CentOS and name their new thing something else, but they didn't, and now they've effectively co-opted CentOS to be something different than it was originally intended to be.

Re:CentOS Project shifts focus to CentOS Stream by bonzini on Tuesday December 08, 2020 @04:25PM (#25351556)
> That in no way explains why they don't continue to have both

Because they don't need it anymore. CentOS Linux or other rebuilds can still exist (just not using the name; I disagree with that but I can understand Red Hat doesn't want its name attached to something that might have large delays in security fixes in the future) if somebody funds it or volunteers to do it, just like CentOS still supports Xen but RHEL does not.

Also for what is worth there have been lots of engineering changes to RHEL in the past couple of years that make nightlies (and CentOS Stream) much more stable than they used to be, especially with respect to regressions. Running CentOS Stream is not going to be like Fedora Rawhide or Debian sid.

Re:CentOS Project shifts focus to CentOS Stream by kbenson on Tuesday December 08, 2020 @05:28PM (#25352315)
>>> it's a change that has strictly technical motives.

I understand the business reasons for doing so. I don't agree with anyone branding this as done for purely technical reasons. Having CentOS Stream may be needed for technical reasons. Stopping CentOS 8 is in no way a technical decision. They are unrelated in any technical sense.

If Red Hat just doesn't want to put resources towards CentOS as it traditionally existed anymore, that's their option, but they deserve any flak they get for taking over an open source project just to extinguish it, since CentOS is in no way really needs to be linked to their Stream product. They could just as easily called it RHEL Stream and said it's free, and it would be a less confusing and more direct funnel of people that want RHEL stability into RHEL subscriptions. Using the CentOS name is just a mind-share grab and screwing over an open source community. They control it so can do it, but that doesn't mean I'm not going to call them out for doing so.

Re:CentOS Project shifts focus to CentOS Stream by bonzini on Tuesday December 08, 2020 @06:59PM (#25353405)
The thing is, Red Hat never considered the distro more than a side effect of providing a base for developing "things" that will run on RHEL. It's even written on the centos.org home page, the distro is not why CentOS existed in 2020. So the fact that users (including myself!!) enjoyed a free distro as a result was not a part of Red Hat's RHEL strategy in any way.
Re:CentOS Project shifts focus to CentOS Stream by kbenson on Tuesday December 08, 2020 @11:15PM (#25355288)
That only makes sense if Red Hat started CentOS. They didn't. The fact that they took control of it then changed it, even the web page, and then are not effectively killed the reason people are using it, is the thing I'm upset about.

If I effectively took control of the EFF and then a couple years later changed the website copy to say that the EFF is a vehicle for litigating cases that kbenson thinks are important, and then actually changed its actions to do so, would you argue the same points? How is this any different? Something that was a net good for many people has been taken over and eventually killed. I think we're all worse off for that.

Re:CentOS Project shifts focus to CentOS Stream by bonzini on Wednesday December 09, 2020 @01:55AM (#25356046)
> I think we're all worse off for that.

I don't disagree.

Re:CentOS Project shifts focus to CentOS Stream by Fnoord on Tuesday December 08, 2020 @10:52PM (#25355130)
IBM is all about expensive support contracts. We cannot afford RHEL. So we went with CentOS 7 and recently CentOS 8. I migrated some machines from 7 to 8. Turns out I did that for one year. My boss wasted precious money there, and it isn't getting us closer to RHEL. My take is its getting us closer to Debian Stable instead. Which is RedHat's loss because a migration to RHEL is then more out of the picture. They know this. They thought of the above. And they are fine with it. That is not a technical decision, as you said. It is a business decision.
Re:CentOS Project shifts focus to CentOS Stream by Koshkin on Tuesday December 08, 2020 @07:19PM (#25353582)
> Because they don't need it anymore.

Isn’t that the crux of the problem? CentOS used to be about “us” (the users), not “them.”

Re:CentOS Project shifts focus to CentOS Stream by bonzini on Wednesday December 09, 2020 @01:57AM (#25356057)
Fire up the wayback machine and find when the centos.org home page changed the mission statement. At this point it started to be about a different "us" than you think, an "us" that doesn't include you and me and presumably doesn't mind CentOS Stream at all.

In particular I started doing that and I got to the Sep 2019-Dec 2019 range, around the time CentOS Stream was launched. At that time this:

> For users, we offer a consistent manageable platform that suits a wide variety of deployments. For open source communities, we offer a solid, predictable base to build upon

was changed to this:

> CentOS Linux is a consistent, manageable platform that suits a wide variety of deployments. For some open source communities, it is a solid, predictable base to build upon.

Re:CentOS Project shifts focus to CentOS Stream by arbitrage on Tuesday December 08, 2020 @05:09PM (#25352072)
> That in no way explains why they don't continue to have both.

They're a business ... are you honestly and in good-faith actually confused about this outcome?

Re:CentOS Project shifts focus to CentOS Stream by zymhan on Tuesday December 08, 2020 @05:39PM (#25352463)
Oh come now, this is Red Hat, a company based on Open Source software. We're not wrong for wanting to hold it to a higher standard. They have benefited from the OSS community just as they have contributed to it.
Re:CentOS Project shifts focus to CentOS Stream by thayne on Tuesday December 08, 2020 @09:44PM (#25354696)
Red Hat didn't make CentOS they acquired it. This is very similar to a large corporation acquiring a smaller competitor, promising to continue supporting it, creating a new, different product using the brand, then dropping the original product.
Re:CentOS Project shifts focus to CentOS Stream by lokedhs on Wednesday December 09, 2020 @12:16AM (#25355609)
Centos was always a free alternative to Redhat. I and many other people used it because it was a way to get the benefits of Redhat without paying for it.

This is a really smart more on their part to get people to stop doing precisely this, and getting them to pay money for RH. Pepople who are not willing to pay (me included) are now rightly annoyed, but we were never their customers in the first place, so we don't really matter.

Re:CentOS Project shifts focus to CentOS Stream by mbreese on Wednesday December 09, 2020 @03:03AM (#25356399)
I’m in the same situation, and I recently had started to plan on upgrading a small HPC cluster from CentOS 7. Before today, I had planned on CentOS 8.

Now? “¯\_(ツ)_/¯“ Probably Debian.

I’ve always used CentOS for clusters, but part of the reason for that is that there are some research packages that support RPM installation, but not deb. At least this gas historically been the case.

If a large amount (maybe even a majority) of users have to switch away from CentOS and RPM packaging, I think we’ll see an acceleration away from RPM as a default option.

So, in that way, I think we do matter, but just not on the balance sheet.

Re:CentOS Project shifts focus to CentOS Stream by the_jeremy on Thursday December 10, 2020 @12:25PM (#25376007)
I disagree. Having a pool of sysadmins / developers who know how to manage CentOS makes RHEL a more appealing alternative than Debian for many companies. The three companies I've worked at have primary used CentOS for development boxes. Shifting to an alternative will change who is familiar with CentOS / RHEL significantly for the worse.
Re:CentOS Project shifts focus to CentOS Stream by zymhan on Wednesday December 09, 2020 @12:15PM (#25360935)
We do matter. We are part of the OSS community that RedHat and IBM benefit from.
Re:CentOS Project shifts focus to CentOS Stream by kbenson on Tuesday December 08, 2020 @05:31PM (#25352356)
I'm not confused at all about that. I was responding to the point that said "it's a change that has strictly technical motives." It's a business decision that's based on profit and positioning and name mind-share, so let's not hide that.
Re:CentOS Project shifts focus to CentOS Stream by nix23 on Wednesday December 09, 2020 @10:55AM (#25359899)
Yes i am, redhat had a really good ride being the trustfully and good guy.
Re:CentOS Project shifts focus to CentOS Stream by norswap on Tuesday December 08, 2020 @08:53PM (#25354416)
I'm wondering, what ties you so strongly to a single OS? Nay, to a single Linux distribution?

I have a hard time imagining, and the only scenarios that come to mind are those where things have gone awfully haywire.

I'm sure I'm missing something. Enlighten me?

Re:CentOS Project shifts focus to CentOS Stream by soraminazuki on Wednesday December 09, 2020 @02:36AM (#25356252)
Software doesn’t run on its own. They rely on bunch of other software that is typically provided by distributions. So unless there’s some magic piece of technology that can take CentOS RPMs and make it work flawlessly on any Linux distro, the entire software industry is suddenly going to have to spend significant amounts of time on repackaging work.
Re:CentOS Project shifts focus to CentOS Stream by joana035 on Wednesday December 09, 2020 @09:29AM (#25358802)
There is alien[1], but I'm not sure if it is flawless, because of slightly different ways of doing things across distros like /etc/default vs /etc/sysconfig for example.

Depending on how good is the source, its usefulness and dependencies, packaging it to Debian is pretty straight forward. The dh_* helpers[2] does the job automatically most of the time. There is also tools for helping with specific languages, like dh-make-golang, dh-python, etc...

[1] https://packages.debian.org/buster/alien

[2] https://manpages.debian.org/testing/debhelper/debhelper.7.en...

Re:CentOS Project shifts focus to CentOS Stream by soraminazuki on Wednesday December 09, 2020 @11:42AM (#25360521)
From the looks of it, alien is just a tool to convert between different package formats and is not a replacement for proper packaging work. Very much like how you can’t take a deb package from different releases of the same distro and expect things to work properly, you can’t just take RPMs from CentOS to a whole different distro and expect things to work.

It’s not only the differences in the packaging format that you have to take care of. There’s also version differences, path differences, dependency handling, and many other stuff to take care of. These are the kind of tasks which can’t be automated away and require non-trivial amount of work.

For organizations that maintain tens, hundreds, or thousands of CentOS packages that spans multiple teams, moving to other distros would be time-consuming and costly. It would certainly pay off if it was driven by technical reasons, but for organizations that are forced to switch by this announcement, this is just pure overhead.

Re:CentOS Project shifts focus to CentOS Stream by nix23 on Wednesday December 09, 2020 @10:53AM (#25359854)
>Java , postgres and tomcat

That's his stack...

>So unless there’s some magic piece of technology that can take CentOS RPMs and make it work flawlessly on any Linux distro,

It's not magic, just alien.

Re:CentOS Project shifts focus to CentOS Stream by soraminazuki on Wednesday December 09, 2020 @11:59AM (#25360739)
> >Java , postgres and tomcat

> That's his stack...

What’s your point and when did I ever mention Java?

> It's not magic, just alien.

Dumping the contents of an RPM archive on random distros is never going to work except for the most simple of cases.

Re:CentOS Project shifts focus to CentOS Stream by nix23 on Wednesday December 09, 2020 @03:10PM (#25364014)
>Dumping the contents of an RPM archive on random distros is never going to work except for the most simple of cases.

That says allot about you packing hygiene, and you don't dump it..that you can do with tar alone. You convert it to a deb.

Re:CentOS Project shifts focus to CentOS Stream by soraminazuki on Wednesday December 09, 2020 @07:20PM (#25367632)
That says a lot about your packaging hygine. With alien, you're essentially dumping the contents of an RPM on hosts it wasn't meant for. It doen't matter that you converted it to deb format along the way, it's the same if you actually install it.
Re:CentOS Project shifts focus to CentOS Stream by nix23 on Thursday December 10, 2020 @05:35AM (#25372162)
MyMy, you have no clue how packages and a package-manager works right?
Re:CentOS Project shifts focus to CentOS Stream by soraminazuki on Thursday December 10, 2020 @02:16PM (#25377275)
Did you ever read a single line of alien source code? It extracts the metadata and files from a package, and re-archives it. That’s about it. It doesn’t handle package version differences, path differences, distro-specific rules, nothing. If your package has dependencies, forget about it because alien won’t retain dependency information it all. In other words, it only “works” for the most simple of cases.

I suggest you read the source code too, because otherwise you can cause real damage if you use it without understanding how it works.

https://sources.debian.org/src/alien/8.95/alien.pl/

Re:CentOS Project shifts focus to CentOS Stream by drewzero1 on Wednesday December 09, 2020 @12:50PM (#25361358)
Took me a minute to figure out where that came from, it was in the GGP comment.
Re:CentOS Project shifts focus to CentOS Stream by norswap on Wednesday December 09, 2020 @12:01PM (#25360756)
But surely if you depend on those packages, you can maybe track (or just collect) their versions, then get the same versions in another package manager (possibly via pinning).

Though my experience with package manager is that dependency management is hell (rife with potential conflicts), so I do see the problem a bit better.

Still, it shouldn't take that long to fix? Like a few days of sprint to setup Nix or something like that?

Re:CentOS Project shifts focus to CentOS Stream by soraminazuki on Wednesday December 09, 2020 @12:34PM (#25361161)
I think it depends on the number of packages that you maintain. If it’s only a handful of packages, the entire process of repackaging, testing, and deployment might be doable in a matter of days. If it’s tens or hundreds of packages that depend on each other, I think that’s a whole different story. If those packages are maintained by different teams, it could take more than an year to complete the transition.

As for pinning packages, that’s only practical if you’re using Nix. As much as I prefer Nix over RPMs, not all of us have the pleasure of using Nix at work. It’s kind of a bummer because Nix packages are so much easier to work with and maintain compared to the competition.

Re:CentOS Project shifts focus to CentOS Stream by nine_k on Tuesday December 08, 2020 @09:52PM (#25354758)
An existing support contract?

If you run a for-profit operation, and downtime is costly, you (or your VP of eng) want a way to pay for immediate assistance from the OS's maker / distributor, when (not if) things go wrong.

Re:CentOS Project shifts focus to CentOS Stream by webmaven on Wednesday December 09, 2020 @09:47AM (#25358963)
> If you run a for-profit operation, and downtime is costly, you (or your VP of eng) want a way to pay for immediate assistance from the OS's maker / distributor, when (not if) things go wrong.

There is a difference between asking "how do I ensure there is one throat to choke when things go wrong?", and "how do I minimize the potential for things to go wrong?"

RHEL is a decent solution if you are trying to answer the former question (or both), but many budget conscious orgs focused on the latter and chose CentOS. Red Hat has now pulled the rug out from under them by trimming eight years off the EOL previously committed to with little notice.

EOL in this case means "no security updates" so even if your org was prepared, technically, to deal with a zero-day for example by rolling out an update in a timely manner without relying on paying a vendor for hand-holding, that option has now been eliminated.

Essentially, you now only get the stability and problem-minimization if you also pay the vendor for support. Otherwise you're stuck with a (relatively) unstable rolling release that will keep your internal teams very busy with a constant stream of minor issues, or potentially trying to roll-your-own updates or backports after the EOL for anything serious.

It's difficult to see this as anything other than a naked, money-grabbing betrayal of users.

Re:CentOS Project shifts focus to CentOS Stream by montjoy on Wednesday December 09, 2020 @10:27AM (#25359453)
I’d recommend using Oracle Linux 8 if you still need 1 to 1 compatibility. At my company I’m recommending Amazon Linux 2.
Re:CentOS Project shifts focus to CentOS Stream by montjoy on Wednesday December 09, 2020 @03:26PM (#25364308)
Not sure why I got downvoted. I’m trying to offer the poster a viable alternative to CentOS 8 with a minimum of changes needed. I get that a lot of people don’t like Oracle (and for good reasons) but really they are the best alternative (and free) EL8 disto now.
Re:CentOS Project shifts focus to CentOS Stream by vbezhenar on Tuesday December 08, 2020 @02:50PM (#25350415)
Consider downgrading to CentOS 7. It should work for 4 more years.
Re:CentOS Project shifts focus to CentOS Stream by soraminazuki on Wednesday December 09, 2020 @05:16AM (#25357119)
Can we take RedHat's word for it, though? CentOS 8 was supposed to be supported until 2029 before this bombshell announcement.

https://web.archive.org/web/20201101131417/https://wiki.cent...

Re:CentOS Project shifts focus to CentOS Stream by rbowen on Wednesday December 09, 2020 @11:17AM (#25360199)
The dates on that page were always dependent on Red Hat. In retrospect, obviously we should have made that clearer, and we will endeavor to do so in the future.
Re:CentOS Project shifts focus to CentOS Stream by p49k on Tuesday December 08, 2020 @04:10PM (#25351397)
Running modern versions of software on CentOS 7 can be a giant pain due to the old version of glibc that’s baked in.
Re:CentOS Project shifts focus to CentOS Stream by AshamedCaptain on Tuesday December 08, 2020 @05:40PM (#25352479)
I was supporting CentOS 6 until about a couple weeks ago.
Re:CentOS Project shifts focus to CentOS Stream by yjftsjthsd-h on Tuesday December 08, 2020 @06:01PM (#25352744)
Then surely you understand what a pain it is.
Re:CentOS Project shifts focus to CentOS Stream by chaz6 on Wednesday December 09, 2020 @03:13AM (#25356444)
An easy way around this problem is to run the software in a docker container with a newer glibc.
Re:CentOS Project shifts focus to CentOS Stream by dijit on Wednesday December 09, 2020 @03:24AM (#25356498)
Docker "works" but has huge performance costs (disk IO) on CentOS7.
Re:CentOS Project shifts focus to CentOS Stream by vbezhenar on Tuesday December 08, 2020 @04:20PM (#25351491)
Java and Postgres will work.
Re:CentOS Project shifts focus to CentOS Stream by nix23 on Wednesday December 09, 2020 @10:57AM (#25359937)
I go and drink some Corretto :)
Re:CentOS Project shifts focus to CentOS Stream by wronglebowski on Tuesday December 08, 2020 @03:46PM (#25351122)
Out of curiosity why did you decide to deploy your application as a VM image? Is there a reason you didn't go with a Helm chart or other container native deployment?
Re:CentOS Project shifts focus to CentOS Stream by chippiewill on Tuesday December 08, 2020 @07:47PM (#25353798)
You're asking why someone who uses CentOS hasn't gone with Helm charts?

That's like polar opposites of stability and bleeding edge.

Re:CentOS Project shifts focus to CentOS Stream by wronglebowski on Tuesday December 08, 2020 @10:03PM (#25354832)
I was curious what drove the decision to deploy that way. I was under the impression most new applications being developed today would choose a more modern deployment method. There’s a lot to maintain in a VM image like that, containers just seem easier to me. Helm chart or otherwise
Re:CentOS Project shifts focus to CentOS Stream by joana035 on Wednesday December 09, 2020 @12:13AM (#25355597)
The amount of maintainance with virtual machines images or container images are mostly the same nowadays.

I will argue that there is less maintenance when handling virtual machines images, because it uses less bandwidth and need less tooling around it comparing with container image based infrastructure.

But in general both are nothing more than the golden image concept.

Re:CentOS Project shifts focus to CentOS Stream by chefkoch on Tuesday December 08, 2020 @06:38PM (#25353136)
>a Helm chart

As a sysadmin i have never heard of this and es tun 800 VMs.

Re:CentOS Project shifts focus to CentOS Stream by hxtk on Tuesday December 08, 2020 @10:22PM (#25354964)
It's a bundle of jinja2 templates for YAML files defining Kubernetes API resources. There's an accompanying application, Helm, which applies values to the templates and handles the interaction with the Kubernetes API.
Re:CentOS Project shifts focus to CentOS Stream by the_duke on Tuesday December 08, 2020 @10:45PM (#25355084)
Helm uses Golangs built-in template language which is definitely not jinja2 compatible.
Re:CentOS Project shifts focus to CentOS Stream by joana035 on Tuesday December 08, 2020 @11:54PM (#25355509)
Basically the same thing: templating.