News for hackers. Stuff that matters.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by em500 on Wednesday December 16, 2020 @03:44PM (#25448329)
I share your disappointment. Out of 150+ comments so far, I believe there has not been a single technical comment about the actual work involved in building a version-pinned RHEL clone.

Without any experience myself (beyond some kernel build maybe 10 years ago), I gathered (from https://wiki.centos.org/About/Building_8) that the majority of work involves manually de-branding the RHEL sources. This apparently can't be automated, as it requires human judgement in which packages/files de-branding is required and in which it might actually break something.

Between major version jumps, e.g. from RHEL/CentOS 7->8 there's apparently also lots of work in getting the build environment up to date.

This raises numerous interesting questions, such as:

1. Where do the upstream RHEL sources live? CentOS sources are in https://vault.centos.org/8.3.2011/BaseOS/Source/SPackages/, but where do they get them upstream? I believe they're only available to RHEL subscribers, does this give RH a way to block clones?

2. Where / what are CentOS's actual build scripts / tools? Is there some howto or writeup how to make a CentOS iso (or cloud-image) on my own PC after downloading the source tree?

3. Do CentOS devs go through the entire manual de-branding exercise with every minor/major update? Presumably they are using some sort of automation/scripting/diffing somewhere. Are these processes/tools available or documented anywhere?

I hope at some point someone with some actual knowledge about this can chime in.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by netzvieh on Wednesday December 16, 2020 @05:00PM (#25449379)
> 1. Where do the upstream RHEL sources live? CentOS sources are in https://vault.centos.org/8.3.2011/BaseOS/Source/SPackages/, but where do they get them upstream? I believe they're only available to RHEL subscribers, does this give RH a way to block clones?

Red Hat publishes it's sources on https://git.centos.org. Those are then used to build CentOS Linux packages.

In the future they'll do their development there, and build CentOS Stream and Red Hat Packages from there.

See also: https://crunchtools.com/before-you-get-mad-about-the-centos-...

> Remember, the source code at git.centos.org is basically read only, downstream code from RHEL. That’s how Red Hat complies with the GPL. Technically we go above and beyond because we are only legally required to provide code to customers, and not required to provide code for BSD/Apache/etc licensed code, only attribution.

Regarding question 2, CentOS has a somewhat custom build system for each major version afaik, for 8 this would be https://koji.mbox.centos.org/koji/

Re:Rocky Linux: A CentOS replacement by the CentOS founder by em500 on Wednesday December 16, 2020 @05:44PM (#25449800)
> Red Hat publishes it's sources on https://git.centos.org. Those are then used to build CentOS Linux packages.

Have you checked if this repo actually works as intended? Because I was wondering if the git repo has RHEL or CentOS sources (or both). So I tried to find out myself instead of just throwing the question out there. It went roughly as follows:

- Let me check the sources of dracut (the RHEL installer) in https://git.centos.org/rpms/dracut. Files: empty, Commits: empty, Forks: empty, Branches and Releases: judging by the names they seem to be CentOS, not RHEL sources. And they're using git.centos.org just as a code dump, not for development. Fair.

- Let's see the actual code.

  git clone https://git.centos.org/rpms/dracut.git
  Cloning into 'dracut'...
  remote: Counting objects: 1673, done.
  remote: Compressing objects: 100% (1632/1632), done.
  remote: Total 1673 (delta 82), reused 1446 (delta 0)
  Receiving objects: 100% (1673/1673), 1.60 MiB | 2.08 MiB/s, done.
  Resolving deltas: 100% (82/82), done.
  warning: remote HEAD refers to nonexistent ref, unable to checkout.
??? There's nothing there.

- Back to the web interface, https://git.centos.org/rpms/dracut/tree/c8 Reading .dracut.metadata, I guess the source is in SOURCES/dracut-049.tar.xz. But browsing the directory https://git.centos.org/rpms/dracut/blob/c8/f/SOURCES lists only patches, not the presumed source dracut-049.tar.xz

- Maybe it's under Releases? Let's check imports/c8/dracut-049-95.git20200804.el8 Clicking the source tree gets me back to the SOURCES dir with only patches, not good. Maybe the floppy-save button? https://git.centos.org/rpms/dracut/archive/imports/c8/dracut... -> 404-Not-Found. https://git.centos.org/rpms/dracut/archive/imports/c8/dracut... -> 404-Not-Found.

I'm not a professional dev, maybe I just don't understand. But is there a way to actually see/browse/download the dracut source code from CentOS 8.3 (let alone RHEL 8.3) from git.centos.org?

Re:Rocky Linux: A CentOS replacement by the CentOS founder by kondbg on Wednesday December 16, 2020 @06:20PM (#25450144)
The git repository stores only the packaging related items (the specfile, the custom patches, etc.). The actual source is stored as a binary artifact that is downloaded by a `get_sources.sh` script.

The build process is documented here: https://wiki.centos.org/action/show/Sources?action=show&redi...

Re:Rocky Linux: A CentOS replacement by the CentOS founder by em500 on Wednesday December 16, 2020 @07:46PM (#25450884)
That did it, thanks. From a cursory glance, it looks like it fetches indeed the RHEL (rather than CentOS) sources. It looks like the mains questions for the clone builders will be how RH is going to provide RHEL code drops for their point releases (and updates) in the future. Since right now there are separate 8 and Stream branches, but presumably the 8 will be discontinued at some point?
Re:Rocky Linux: A CentOS replacement by the CentOS founder by quicklyfrozen on Wednesday December 16, 2020 @06:29PM (#25450211)
The RPM SPEC file in that repo will have a pointer to the actual upstream sources for the package. This is a typical scenario -- they are not re-hosting all of the sources to build a Linux distro, just the build steps needed to pull, patch, and build upstream sources.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by em500 on Wednesday December 16, 2020 @07:02PM (#25450491)
So where do I actually get this upstream (RHEL8) source of say dracut? Because I was reacting to the comment "Red Hat publishes it's sources on https://git.centos.org. Those are then used to build CentOS Linux packages."
Re:Rocky Linux: A CentOS replacement by the CentOS founder by quicklyfrozen on Wednesday December 16, 2020 @08:11PM (#25451087)
rpmbuild will download the sources listed in the spec file, apply the patches, and execute the build instructions to produce RPM and SRPM packages. The SRPM will contain the "as built" source tree.

For dracut, the spec file defines the source as http://www.kernel.org/pub/linux/utils/boot/dracut/dracut-049...

Re:Rocky Linux: A CentOS replacement by the CentOS founder by tuldia on Wednesday December 16, 2020 @07:57PM (#25450970)
Seems that dracut upstream is hosted at kernel.org.

This is the regex used in the watch file for Debian to fetch new upstream tarballs: http://www.kernel.org/pub/linux/utils/boot/dracut/dracut-([\...

Re:Rocky Linux: A CentOS replacement by the CentOS founder by joseph on Wednesday December 16, 2020 @04:31PM (#25448998)
RHEL source RPMs can be downloaded from http://ftp.redhat.com/pub/redhat/linux/enterprise. According to your link, CentOS doesn't use the source RPMs since 7 but uses git repos instead. I don't know where the git repos are located, however, or if it is still possible to build the whole OS from just the SRPMs.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by mroche on Thursday December 17, 2020 @12:22AM (#25452793)
The FTP server stopped shipping actual distro SRPMs with 7. The 8Base is for addons and OpenStack. See https://news.ycombinator.com/item?id=25452762

And in terms of rebuild ability, RHEL is not a self-hosting distribution. There are missing dependencies and packages that need to be built in order to create a fully serviceable distribution as Red Hat does not ship those packages.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by soperj on Wednesday December 16, 2020 @04:13PM (#25448771)
>1. Where do the upstream RHEL sources live? CentOS sources are in https://vault.centos.org/8.3.2011/BaseOS/Source/SPackages/, but where do they get them upstream? I believe they're only available to RHEL subscribers, does this give RH a way to block clones?

How'd they used to do it then? I assume they just became a subscriber?

Edit: just thought about it, couldn't they get them from CentOS? Would be pretty funny.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by em500 on Wednesday December 16, 2020 @04:24PM (#25448905)
From what I understand[1], up to RHEL6, Red Hat released their sources on public ftp, but after that they became customer/subscriber-only.

I would have thought that the GPL ensures that the customer can then freely redistribute them, but I've read that companies can still make that option very unattractive through other means (e.g. terminating the customer contract, mingling sources with proprietary stuff and making them hard to disentangle). I don't know if RH is playing such games, but the complete lack of non-gated RHEL7 and 8 source code on the web gives some strong hints.

edit: relevant thread: https://github.com/rocky-linux/rocky/issues/4

[1] https://lwn.net/Articles/603865/

Re:Rocky Linux: A CentOS replacement by the CentOS founder by not2b on Wednesday December 16, 2020 @06:44PM (#25450340)
The difficulty is that RHEL isn't a GPLed work. It's a mix of free software under various licenses and logos and other oddball things that are protected by trademark or copyright and are non-free (the non-free part is mostly icons and images identifying the system as RHEL). There isn't a lot of that, but it all has to be separated out to be legal to redistribute. If you go through all the work to do that, you wind up with the equivalent of CentOS. But RHEL's customers don't have the incentive to do that work.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by macksd on Wednesday December 16, 2020 @04:35PM (#25449066)
>> I would have thought that the GPL ensures that the customer can then freely redistribute them

I'm actually surprised Red Hat has typically shipped SRPMS in bulk to it's customers. I think it's rare that customers would use them, and the GPL allows Red Hat to be far less accomodating. It allows you to charge for each copy of the source code you convey, it doesn't need to be in such a convenient form, it doesn't need to be available on-demand, it just needs to be available on-request.

Once a customer has it, yes - the GPL allows them to hand it to Rocky Linux and let them run with it. But I think the community has been fortunate so far just to get the distro sources they have in the way they've gotten them.

edit: Perhaps "fortunate" isn't the right word, since Red Hat benefits from the community and owes the community some reciprocation. I'm just saying that legally, if Red Hat wanted to be bigger douche bags, the GPL gives them some space to do so. And I'm glad they haven't fully taken advantage of that before.

Re:Rocky Linux: A CentOS replacement by the CentOS founder by not2b on Wednesday December 16, 2020 @06:51PM (#25450398)
The requirement to distribute source means you have to provide the exact source for any binary that you distributed, for three years. And the source is defined as " the preferred form of the work for making modifications to it". A pointer to the upstream tarball plus a poorly organized directory with 58 patches in it isn't "the preferred form of the work ...". SRPMs are an easy way to make sure that you got it right, that the source really corresponds to be binary. Attempting to add speed bumps risks getting it wrong and providing a way for pissed-off developers to make trouble.
Re:Rocky Linux: A CentOS replacement by the CentOS founder by jkaplowitz on Wednesday December 16, 2020 @11:05PM (#25452322)
I think the alternative which the other commenter had in mind was simply providing desired SRPMS to customers reactively upon request, possibly with a nonzero charge to cover, rather than proactively providing them to all customers in bulk. The GPL certainly allows this, as opposed to a tarball and patches which you're right is inadequate.

Interesting wrinkle in the GPL's written offer option: the offer must be valid for "any third party", not only direct customers. But, it's certainly possible for the included written offer to reference an email address or website that is unique to each paying customer, such that valid request coming from a third party would be traceable to the intermediate paying customer whose support contract would then be canceled by Red Hat.

Does anyone know if Red Hat does this kind of watermarking of the GPL written offers, or if a licensee could share the offer anonymously, leak-style, and not get caught by the support contract people?

Re:Rocky Linux: A CentOS replacement by the CentOS founder by macksd on Thursday December 17, 2020 @10:03AM (#25456150)
I don't disagree, but I've seen other open-source companies view the tarball as their source release and I wouldn't be the least bit surprised if you could at least make enough of an argument to get a judge to go along with it long enough to make it prohibitively expensive to reverse.