News for hackers. Stuff that matters.
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by derefr on Sunday December 24, 2023 @05:02PM (#38757354)
What was ever the point of the $PREFIX/bin vs $PREFIX/sbin division, anyway?

I understand the historical reasoning for the / vs /usr vs /usr/local split —

• / was a mount of the machine vendor's installation-engineering-provided bring-up tape — a tape that could usually be safely unmounted once boot was complete and the kernel + persistent daemons had loaded into RAM. (We still have something akin to this in Linux land in the form of the initramfs!)

• /usr was a read-only-mounted OS distribution, either spooled onto the same tape drive that had been reading the rootfs tape a moment before, or (later) over NFS from a high-read-IOPS shared disk server, where in either case the mountpoint was expected to never go away while the OS was running, and therefore the loader could spawn binaries from here using flushable memory-mapped disk IO (allowing far more running processes to fit within the machine's limited memory — important for time-sharing!);

• /usr/local was a separate site-specific mount of stuff added by the local sysadmin — often created as a separate "tacked on the end" partition on the OS distribution tape (you could just attach more tape on the end of the reel, and then format it!) Later, once machine-local hard disks were a thing but Unix systems were still generally mounting /usr over NFS, /usr/local evolved to be the read-only partition of the machine-local HD (and such systems gained a matching read-write local-disk partition, /var.)

But $PREFIX/bin and $PREFIX/sbin were, as far as I know, always just regular directories under the same $PREFIX-level mountpoint, never separate mountpoints of their own.

The only difference I can even think of between a $PREFIX/bin and a $PREFIX/sbin dir, is that the $PREFIX/sbin dir wasn't in $PATH for most users. But surely that was never a real line of defense for security?

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by kristopolous on Sunday December 24, 2023 @05:13PM (#38757460)
/usr was initially for users. See Kernighan's Unix for beginniners from 1974: https://archive.org/details/unix-for-beginners/page/n5/mode/... or this 1982 text from Bell Labs: https://archive.org/details/unixsystem0000bour/page/10/mode/...

Even in 1994 this was still common, https://archive.org/details/learningunix0000gard/page/48/mod...

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by derefr on Sunday December 24, 2023 @05:23PM (#38757563)
Interesting. I guess 1974 was before "distributions" like BSD even existed, when the OEM bring-up tape was the only OS tape.

But why does Kernighan, on that same page, mention /usr/bin already being a thing? If /usr is what we would today call /home, why does it have a bin subdir?

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by kristopolous on Sunday December 24, 2023 @05:31PM (#38757641)
$ cat /etc/passwd

Look at row #3.

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by derefr on Sunday December 24, 2023 @05:43PM (#38757759)
That only moves the question back a step. Why have a bin user that owns all the userland binaries and keeps them directly under its own home directory? What does that accomplish? Something to do with per-user quotas? Or making `ln /usr/bob/a.out /usr/bin/newcmd` cheap?

I note that there's also a bin group. Was /usr/bin mode 1775, and anyone who should be allowed to install binaries (and update/remove their own installed binaries) was made a member of the bin group?

Also, I note that there's no sbin user. (There is, however, a sys user. But only on Linux, not on BSD...)

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by kristopolous on Sunday December 24, 2023 @06:12PM (#38757959)
sbin is a SYSVR4'ism ... it's complicated. /etc had a bunch of binaries in it like fsck. MACH systems had that as well (well technically it was linked to /private/etc which then linked the binaries to /usr/etc - here's an openstep screenshot I just took: http://9ol.es/usr-etc.png). It's /etc/mkfs /etc/route ... they got moved to /usr/sbin/ and /sbin in R4 - by then, SYSV systems were using /home and the initial purity of every owner of every process being an equivalent class user had been broken by pragmatism.

The invention of /sbin/ /usr/sbin is all R4, see https://archive.org/details/unixsystemvrelea0000unse_t8i3/pa...

Unix 1980s history is a complicated mess ... that's why there were multiple conflicting standards groups that tried to fix it. The X/Open group had the "single unix specification" which is different from POSIX and say, the Open software foundation (which did OSF/1). Here's a nice crash course: https://en.wikipedia.org/wiki/Single_UNIX_Specification

Explaining the history with the motivations is probably around 10,000 words. AFAIK it hasn't been written. I'd be one of the handful of wackos who'd actually do that and I haven't seen it.

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by db48x on Sunday December 24, 2023 @08:44PM (#38758807)
It let them add a second hard drive after the first one filled up (5MB each, as I recall).
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by derefr on Tuesday December 26, 2023 @02:42PM (#38775268)
You misunderstand. My question was why they bothered to make a real "bin" user to have /usr/bin as its homedir; when, if it had just been purely an issue of shoving some binaries over to the /usr drive, they could have just created a /usr/bin owned by root:root and leave it there; or, if they really cared, created a /usr/root for "a secondary root-user homedir that lives on the /usr drive" and then made a /usr/root/bin.
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by db48x on Tuesday December 26, 2023 @08:11PM (#38777932)
That I don’t know. My guess is that it came later, so that no new user could be called “bin” and thus inherit ownership of those files. There are some historical sources that might shed light on the matter, but someone would have to do some work.
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by rwmj on Sunday December 24, 2023 @05:11PM (#38757444)
System V introduced /usr/sbin for storing loosely defined "system binaries", meaning admin tools, daemons and such like. The idea was that they wouldn't appear on the ordinary user's $PATH (and confuse them, I suppose).
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by wredue on Sunday December 24, 2023 @05:18PM (#38757514)
So what use $HOME/.local/bin for these days?
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by sph on Sunday December 24, 2023 @05:28PM (#38757614)
AFAIK ~/.local/bin is not in any standard. I create this directory manually after installation and add it to my PATH because it makes sense to me, but it is not mandated by the FHS/FreeDesktop/XDG.

None of the mainstream distributions create nor manage that directory for you.

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by JoshTriplett on Sunday December 24, 2023 @05:31PM (#38757645)
> AFAIK ~/.local/bin is not in any standard. I create this directory manually and add it to my PATH because it makes sense to me, but it is not mandated by the FHS/FreeDesktop/XDG.

Quoting https://specifications.freedesktop.org/basedir-spec/basedir-... :

> User-specific executable files may be stored in $HOME/.local/bin. Distributions should ensure this directory shows up in the UNIX $PATH environment variable, at an appropriate place.

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by sph on Sunday December 24, 2023 @05:32PM (#38757657)
TIL.

Still, I haven't seen any distro create or put it in PATH in the 24 years I've been running Linux.

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by derefr on Sunday December 24, 2023 @05:53PM (#38757827)
Most of the $XDG_FOO_DIR dirs are just-in-time created with a `mkdir -p` when something wants to put stuff under them. Given that you're running a program that manipulates the XDG dirs, that program must already exist somewhere else on your system — so it probably doesn't need to write any binaries. It's already installed!

Also, language package ecosystem tooling — the one thing most people touch that could end up installing user-local binaries — is generally just lame and doesn't obey XDG. For just one example, "per-user global" installation of Ruby gems (`gem install --user-install foo`), should properly install the gems under ~/.local/libexec/gem, and then symlink the binstubs of said gem into ~/.local/bin. But it doesn't; it just dumps everything into its own ~/.gem dir and then expects you-the-user to figure out what dir under there to add to your $PATH. Same applies to Rust, Go, Python, and every other language I'm aware of.

One thing you can do that will almost-assuredly use ~/.local/bin/, though, is to intentionally point an autotools configure script at ~/.local as its --prefix. `make install` will then dump binaries into ~/.local/bin.

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by JoshTriplett on Sunday December 24, 2023 @06:31PM (#38758083)
I'd love to see more language tooling install to ~/.local/bin. I think quite a bit of it predates the widespread use of that XDG directory, so adopting it now would require a migration. I think it'd be worth it, but I also think that's why it hasn't happened yet.
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by znpy on Sunday December 24, 2023 @05:24PM (#38757571)
Eh, afaik /usr/local was for storing stuff installed by the local sysadmin and not distributed by the vendor… ~/.local the was for stuff compiled by each individual user and installed locally (in their home directory).

The idea is that if a user needs some kind of software and is able to build it autonomously then they can set $PATH and other environment variables to make it available to the session. Please note that in ~/.local there are bin, man, lib and other directories, so you can pretty much build, install and run what you need without bothering your sysadmin :)

These days… some package managers might put stuff in there so that they can work without needing root permission and without messing up with the general system.

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by Phrodo_00 on Sunday December 24, 2023 @05:29PM (#38757622)
Shared computers are still a thing (in computer labs or servers) so that use of .local is still around.
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by JoshTriplett on Sunday December 24, 2023 @05:30PM (#38757629)
Sometimes, the binaries in /sbin were statically linked, to make them more likely to work when attempting to recover from disk corruption or similar.
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by ajross on Sunday December 24, 2023 @05:44PM (#38757762)
> I understand the historical reasoning for the / vs /usr vs /usr/local split —

That was retconned, actually. My understanding is that the original "/usr" was a second RK05 disk pack added to an early system at Bell Labs for storing stuff owned/created/managed by "users" instead of the system folks. Space was precious in those days and RAID was still a decade in the future. You got your 2MB and had to decide where to mount it and what to put there.

Early Unix filesystems were an ad-hoc wreck, basically. Sun cleaned this up a lot as they moved (or tried to move) to NFS as the primary filesystem for workgroup installations, which is where we get the modern ideas from that you are tagging "historical". I believe (but am not sure) that "sbin" hails from this era.

Then Red Hat came by another decade later and further symmetrized the rules, but in the process essentially obsoleted the distinction between "/" and "/usr", etc...

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by eesmith on Sunday December 24, 2023 @05:20PM (#38757529)
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s16.htm... says:

> Utilities used for system administration (and other root-only commands) are stored in /sbin, /usr/sbin, and /usr/local/sbin. /sbin contains binaries essential for booting, restoring, recovering, and/or repairing the system in addition to the binaries in /bin. ...

> Originally, /sbin binaries were kept in /etc. ...

> Deciding what things go into "sbin" directories is simple: if a normal (not a system administrator) user will ever run it directly, then it must be placed in one of the "bin" directories. Ordinary users should not have to place any of the sbin directories in their path.

I don't think it was a security thing, more a usability thing? Why see "reboot" on your path when you can't use it?

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by PrimeMcFly on Sunday December 24, 2023 @05:33PM (#38757667)
> $PREFIX/bin vs $PREFIX/sbin division

Why did you go to so much effort to avoid typing 'usr'?

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by derefr on Sunday December 24, 2023 @05:41PM (#38757739)
Because /bin, /sbin, /usr/local/bin, and /usr/local/sbin all also exist?

(And bevcause GNU autotools uses the variable $PREFIX to abstract away the differences between these — you can `./configure --prefix=/usr` or `./configure --prefix=$HOME/.local` if you like.)

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by PrimeMcFly on Sunday December 24, 2023 @06:17PM (#38757993)
It was just weird to me. The title specifies /usr as the prefix. If you meant the division in general you could have just left off $PREFIX altogether.
Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by derefr on Monday December 25, 2023 @04:16PM (#38766202)
"bin vs sbin" doesn't read right as English — it's almost-but-not-quite like the words are proper nouns that should be capitalized, but instead, it's that they're missing a symbol to indicate that they're paths. The vagueness would become problematic if I wanted to also use "bin dir" in the same paragraph to refer to binary-containing directories generally (i.e. both bin and sbin.)

And the fact that "bin dir" is used that way, to group together both bins and sbins, is is why I didn't say "bin dirs vs sbin dirs." (Note how the prior sentence reads really oddly?)

Putting them each in quotation marks — "bin" vs "sbin" — requires some annoying footwork if you expect you'll need to put larger sections of the paragraph in quotation marks. Most editorial guidelines discourage writers from using quotation marks to mention jargon terms for this reason — especially if the terms will be coming up more than once in a piece.

"/bin vs /sbin" sounds like I'm talking specifically about the ones under /, rather than all of them.

"./bin vs ./sbin" just sounds like I got confused and am talking about something that isn't actually the subject of the parent post.

".../bin vs .../sbin" is pretty good — but "..." here is just doing the same thing "$PREFIX" does but with less guidance as to how it should be interpreted — less ability to Google what it means.

Trust me, I considered every option before settling on writing something as silly as "$PREFIX/bin". "$PREFIX/bin" is weird, but clear. And without an elegant best option, the "weird but clear" option is generally the one you should choose in technical writing. People can't generally call you 20 years later to help parse through a sentence that conflates three different usages of a term, but they can just Google any weird (but well-documented) jargon you've used.

Re:Fedora 40 Plans To Unify /usr/bin and /usr/sbin by PrimeMcFly on Monday December 25, 2023 @04:28PM (#38766289)
I think "bin vs sbin" reads fine as English given the context of the article, and if you wanted to be more exact you could just use single quotes. I think "bin dir" ad "sbin dir" work fine as well because of the context of the article.

That's just me though. I appreciate the thought you put into your decision as well as your detailed explanation, thank you. I wasn't expecting a detail explanation, but I am always interested in people's reasoning when it's different from mine.