News for hackers. Stuff that matters.
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by peterwwillis on Wednesday March 18, 2020 @09:32AM (#22616429)
And this has led to de facto standards, like '#!/bin/sh' is a 'standard' shell shebang, but not POSIX; if you want your script to be portable, you have to use '#!/usr/bin/env sh'. env is forever in '/usr/bin' probably because it first came about after new tools started moving to '/usr'. And I don't think env's file path is even in POSIX, it's just a de facto standard.

Another fun fact is that execve() (which executes shebangs) only allows a single argument to the shebang program. Meaning if you use '#!/usr/bin/env sh', you can't specify arguments to 'sh' in the shebang. In any case, the name of the file being executed is appended as an additional argument at the end of the shebang.

Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by _kst_ on Wednesday March 18, 2020 @07:06PM (#22622423)
env hasn't always been in /usr/bin. I've worked on systems (probably SunOS 4.something) that had /bin/env but not /usr/bin/env. But you're unlikely to run into any reasonably modern Unix-like system that doesn't have /usr/bin/env .

See also my Stack Exchange answer on the advantages and disadvantages of the "#!/usr/bin/env ..." convention: https://unix.stackexchange.com/a/29620/10454

Quick summary: The advantage of "#!/usr/bin/env INTERP" is that it will use the first INTERP in the user's $PATH. The disadvantage is that it will use the first INTERP in the user's $PATH.

Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by catalogia on Wednesday March 18, 2020 @04:01PM (#22620663)
If neither /bin/sh nor /usr/bin/env are standard but both are defacto standards, then where is the relative value of `#!/usr/bin/env sh` over `#!/bin/sh` ?
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by _kst_ on Wednesday March 18, 2020 @07:07PM (#22622437)
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by duckerude on Wednesday March 18, 2020 @04:51PM (#22621114)
As a data point, my phone has /bin/sh but not /usr/bin/env. But not many shell scripts need to target Android, probably.
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by saagarjha on Wednesday March 18, 2020 @05:17PM (#22621344)
env is part of POSIX, so I guess your phone just doesn’t make the symlink. (If it’s busybox, perhaps it still has the applet compiled in?)
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by duckerude on Wednesday March 18, 2020 @06:13PM (#22621862)
It has /bin/env, it just doesn't have /usr.
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by beefhash on Wednesday March 18, 2020 @09:47AM (#22616572)
> And I don't think env's file path is even in POSIX, it's just a de facto standard.

You would be correct about that. And that sucks.

Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by kyuudou on Wednesday March 18, 2020 @12:57PM (#22618727)
There's some debate about this if I recall correctly from the last deep dive I did about the pros and cons of it.

If I am writing a script that needs to be that portable, then I'll use #!/usr/bin/env sh but otherwise /bin/sh will work in all the cases I need it to.

Since bash's path might vary more, I might consider calling env to locate it then more often.

Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by naniwaduni on Wednesday March 18, 2020 @03:12PM (#22620173)
If you're writing a script that needs to be that portable, you're doomed anyway and you might even find the autoconf manual's "Limitations of Shell Builtins" section useful.
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by oat_bravo_nap on Wednesday March 18, 2020 @04:21PM (#22620840)
If you are using /bin/sh you need to check sure it is a POSIX shell script and not using bash features [1]. On Debian and derivatives /bin/sh is ash not bash.

[1] https://linux.die.net/man/1/checkbashisms

Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by lmm on Wednesday March 18, 2020 @10:16AM (#22616876)
I'm not aware of anywhere that /bin/sh won't work. /bin/bash is extremely non-portable though.
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by dwheeler on Wednesday March 18, 2020 @12:50PM (#22618650)
POSIX does not specify the path of anything. It says that "sh" is a shell, but doesn't say that "/bin/sh" is that shell. See: https://pubs.opengroup.org/onlinepubs/9699919799/

I believe that in Solaris before Solaris 10, /bin/sh was the original Bourne shell (which didn't comply with POSIX), and in Solaris 10 it's ksh (which isn't either, but is way closer): https://unix.stackexchange.com/questions/538844/does-posix-g...

Also, I don't think that "#!" is specified in POSIX. It's extremely common, it's just not in the specification. This is similar to find's "-print0" option, which also isn't in the POSIX specification but basically everyone implements it.

If you want a specification of "where things go", the closest one in practice (and one that most modern distributions try to follow) is the Filesystem Hierarchy Standard (FHS): https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html

Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by LukeShu on Wednesday March 18, 2020 @02:56PM (#22620026)
(providing more detailed references)

POSIX specifically calls out that it doesn't guarantee that the path is "/bin/sh":

    107271  Applications should note that the standard PATH to the shell cannot be assumed to be either
    107272  /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by
    107273  getconf PATH, ensuring that the returned pathname is an absolute pathname and not a shell
    107274  built-in.
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/s...

POSIX mentions "#!" several times, but doesn't require it; saying things like "on systems that support executable scripts (the "#!" construct)".

As the lack of a hard-codable path is frustrating, it recommends setting the shebang at install-time:

    107279  Furthermore, on systems that support executable scripts (the "#!" construct), it is
    107280  recommended that applications using executable scripts install them using getconf PATH to
    107281  determine the shell pathname and update the "#!" script appropriately as it is being installed
    107282  (for example, with sed). For example:
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by JdeBP on Wednesday March 18, 2020 @03:56PM (#22620598)
As I wrote at https://unix.stackexchange.com/a/567405/5132 , I have a strong suspicion that Arch Linux dropping the FHS is the tipping point, and adherence to the FHS is now the exception not the norm.
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by squiggleblaz on Thursday March 19, 2020 @06:22AM (#22626134)
Why would Arch dropping the FHS be the tipping point? Surely Arch represents hackers and not mainstream Linux users. In much the same way I wouldn't cite Linux users are a tipping point in any computer system convention (but Mac OS or Android or Windows), I wouldn't cite Arch in a Linux system convention (but Debian or Ubuntu or Red Hat). Why do you consider Arch as the tipping point?
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by JdeBP on Thursday March 19, 2020 @11:51AM (#22628608)
Because it would be, if my suspicion proves to be the case, the point where adherence to the FHS tips over from being the norm to the exception. MacOS, Windows, and who Arch users are have nothing to do with it. How many distribution creators have made each of the various choices, does. Arch's decision to use systemd had some influence (in both directions, c.f. Hyperbola Linux). Arch's decision to abandon the FHS in favour of the systemd file-hierarchy may well do likewise.
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by toast0 on Wednesday March 18, 2020 @12:20PM (#22618297)
/bin/bash is popular in Linux systems, but not BSD or Mac or probably other Unixes which may not always have bash and if they do have it, it will be somewhere else. Also, a number of niche-ish Linux distributions are trying to replace bash with something else. It's no longer as portable as it once was.
Re:Understanding the bin, sbin, usr/bin, usr/sbin split (2010) by bobbyi_settv on Wednesday March 18, 2020 @05:18PM (#22621353)
It's not just about whether it exists but also the version.

For example, on my (Mac) laptop, /bin/bash is bash 3.2.57 (released in 2007), while "/usr/bin/env bash" gets the version installed by homebrew which is 4.4.19 (released in 2016).