> Whelp, wrong right off the bat.
It really is a Google guide, tailored to Google engineers. If they know for sure that bash is available on any *n?x machine, then why not? It's just a local policy.
And if it's that much of a problem, the other places where you get bash instead of POSIX sh from Google (chromium?) the license clearly allows you to rewrite said script to suit your needs.
All other points are valid though.
Note that I chose to write all my scripts in POSIX (e.g. https://gitlab.com/moviuro/moviuro.bin).
And posted in a public forum, which is why I made sure to protest its adoption by the broader public.
It's really not. It's only present on some desktop Linux distributions. It's not present on almost any other OS or anywhere in the embedded space. POSIX shell, on the other hand, is everywhere.
I mean, what are the use cases? Where universality is important (so let's say when someone embeds scripts in Makefiles), I understand the need for POSIX, but other that I can't even come up with a good use case for having sh as default. (Maybe to save space, you ship the embedded device or Docker container with sh, sure, but then what kind of script you'd want to run there? Okay, maybe all of them, but then even on alpine installing bash is one quick command.)
Busybox is a thing. Many embedded devices don't have the luxury of installing Bash–you're stuck with what you've got.
I think you answered your own question here.
I can elaborate, though. bash isn't portable, it uses Glibc-isms. It has to be explicitly ported to new platforms. I also pointed out elsewhere that the value-add of bash is dubious at best - if you need to use bashisms you're better off not writing a shell script at all. So really, the question is: why do you need bash? You have to justify using non-standard tools, not the other way around.
And I feel the same with targeting POSIX sh.
Unless you have a really good business niche - like the aforementioned ftp script, use a proper language.
Yes, sysadmins were opposed to installing Bash. But if your core product depends on Bash and only on Bash, maybe I'd be opposed to everything about that deployment too.
And at the same time, I understand that there are IT tasks well suited to be solved with a script. But those don't seem to be the ones that require uber-portability.
If I need to setup something on a fleet of VMS-es, I'll use whatever I can get away with (including a lot of controlled substances to handle the dread of handling VMS), and won't think about what happens if we port this to HP-UX in the year 2525.
Indeed! The significance of this, I believe, is underappreciated.
During 2008-2015, when I used to work for a network security company, we had a file transfer script[1] for Unix and Linux systems. It was written entirely as a shell script. It began as a simple wrapper around the scp and sftp commands for which a shell script was appropriate. The fact that every Unix and Linux system comes with a shell meant that we could ship a single file to all target Unix and Linux systems without requiring the administrator to install any new packages. The alternative was writing the file transfer functionality in C or C++ which would have required us to build and test the solution for every possible Unix system that our customers used or writing it in Java which would have required us to require the customer to install JVM on their systems. Many customer admins were averse to installing any additional packages on their systems. Given these constraints, a shell script was a good solution.
Unfortunately, the initial shell script relied on several bashisms.[2] Some customers began complaining that their Solaris or AIX system did not have bash, and they needed the script to work with Bourne shell or Korn shell. The customers refused to install bash. So I decided to clean up the script, remove all bashisms from it, and ensure that only POSIX specified features were used, only POSIX specified commands were invoked with POSIX specified options only (with the exception of the scp and sftp commands of course because that was the primary reason why the customers wanted the script).
I am not fond of bashisms. If I need bashims like arrays or regex substitutions in parameter expansions, it's a sign that the script is becoming complex and I switch to a modern programming language. These days, when I write a shell script[3], I restrict myself to POSIX.2001 (2004 edition)[4], write unit tests to exercise every line of code, run the unit tests with bash, ksh, zsh, dash, posh, and yash on Debian, run the tests with bash, ksh, and zsh on macOS, and use Kcov[5] to measure test coverage.
[1]: https://community.rsa.com/docs/DOC-53124
[2]: https://mywiki.wooledge.org/Bashism
[3]: https://github.com/susam/vimer
You can test ksh93 and the MirOS Korn shell on Debian. For the Heirloom Bourne shell, though, look to FreeBSD, which has everything except the Watanabe shell.
* https://svnweb.freebsd.org/ports/head/shells/heirloom-sh/
* https://svnweb.freebsd.org/ports/head/shells/pdksh/
* https://svnweb.freebsd.org/ports/head/shells/ksh93/
* https://svnweb.freebsd.org/ports/head/shells/mksh/
* https://svnweb.freebsd.org/ports/head/shells/oksh/
* https://svnweb.freebsd.org/ports/head/shells/dash/
* https://www.freebsd.org/ports/shells.html
I appreciate what this sort of thing involves, as you can infer. (-:
What about POSIX shell makes it superior for scripting?
Or to put it the other way around: if you find yourself writing a POSIX shell script and find that you'd like to use more advanced features then one of two things can happen:
- You need portability to SunOS 4, so you suck it up and keep hacking your script because bash is not an option anyway;
- You don't need portability, so then why not use a proper programming language like python, perl or ruby rather than upgrading from crappy sh to slightly-less-crappy bash?
If you want a policy against shellscripts growing out of hand have a policy against that, not that you must use the lowest common denominator even though you have Bash everywhere.
And I'm saying this as someone who almost exclusively writes POSIX sh for portability reasons, but my software isn't meant to target just Google's infra.
But anyway my argument was more about my opinion about a developer talking for myself, not a company style guide. Life's too short to learn two shell dialects so I can either learn POSIX which will work basically everywhere and will let me write any simple script just fine or write Bash whose added complexity I probably won't need or even want.
The fact that I'm an embedded dev probably biases my view though, I generally target environments where bash isn't an option anyway.
#!/usr/bin/env python
Rather than the more portable: #!/usr/bin/env python3
Notably this causes problems for people on Arch Linux, and in the near future, Fedora. This post is presented as a style guide that others should adopt. Google may have bash ubiquiotously available internally, but their approach should not be taken as a model for others. Google's approach has caused real problems for their software's portability in exchange for dubious value-adds.>What about POSIX shell makes it superior for scripting?
To address this specifically, I have written a blog article on the subject:
https://drewdevault.com/2018/02/05/Introduction-to-POSIX-she...
The main points are:
- bash is far from as ubiquitous as some people seem to think it is
- sh is formally standardized and has multiple competing implementations, whereas bash is defined by its implementation and there is only one
- sh is perfectly competent and easy to target, bash's value add is questionable
- By the time you need the things bash offers you might as well not use a shell script at all
I wrote that script exactly because of the issue you described, and I got tired of changing what /usr/bin/python is symlinked to constantly. With that script, I just run `py2 gn gen out/Debug` or whatever, and my /usr/bin/python stays intact so everything else which expects python3 works, and only the `gn` process and all subprocess sees python2.
This definitely doesn't excuse Google's behavior, but I don't expect it to be fixed soon. My bug report about it (https://bugs.chromium.org/p/webrtc/issues/detail?id=7376) was closed as wontfix (though I'm not entirely sure the person who closed it understood that I was asking for the python scripts to explicitly use python 2, not for them to port the scripts to python 3).
People have been on 2.7 for so long that a breaking change might be the only thing to get them to move over.
As Chromium is open source, I'm surprised they haven't really explained how people outside of the Googleplex should handle this.
Seems like they're left to their own devices.
/bin/sh
is standardized is incorrect, though I don't know how many actual systems fail to have it present. Source: https://news.ycombinator.com/item?id=17069408 (I checked the 2018 version of the spec, and it says the same).Its not. The software was written, and maintains, backwards compatibility with systems that predate or partially predate the spec. The spec was written to be backwards compatible with those systems. Using the backwards compatibility option is correct if you wish to maintain backwards compatibility. Consider that Arch also technically violates Pep 394,
> for the time being, all distributions should ensure that python, if installed, refers to the same target as python2
We shouldn't update the shebangs in all of our software ever to accommodate a misbehaving, spec-non-compliant distro, should we?
I mean, standards aren't necessary; interpreters can absolutely be snowflakes. But multiple implementations is a sign of strength, and intentionally targeting the bugs of one implementation is short sighted, if sometimes required.
There is no standard to hold bash to. Any strange behavior of it might be decided as by design and kept forever (and replicated by anything which tries to be bash compatible) or it could be determined to be a bug and, when fixed, break the scripts which worked around it or depended on it. There's no way to know how this is going to shake out because there is no objective specification for the language. At least when there's a bug in one of these competing sh implementations that you're so worried about, it's objectively a bug and can be fixed.
But I didn't think about the bug vs. by-design issue and how that is remedied by a well-defined standard. That does seem to be a factor in favor of using a standardized language. But it doesn’t seem different in principle than depending on any software. It is liable to change and leave you with a difficult decision of forking the library or modifying your code that consumes it. Given bash’s ubiquity, it seems reasonable to expect a good level of stability.
If you only target standards, the lowest-common-denominator of functionality, you will be poorer for it. Linux, for example, has all sorts of performance and functionality enhancing features over what POSIX provides.
In this case, you won't. I addressed this a few comments ago:
>By the time you need the things bash offers you might as well not use a shell script at all
Yes. GCC will die eventually.
Bash is a de facto modern standard.
The only new purely POSIX shells I know are for embedded stuff (dash) and that space is neither fast moving nor in dire need of a better shell, since most modern shell features target interactive use. Or they make coding easier and/or safer, in which case we're back to point 1, with the POSIX incompatibility.
Heck, Microsoft barely managed to supplant cmd 1-2 years ago, and that's in a non-standard, proprietary environment by what is arguably the biggest commercial platform maker in the world.
Though I agree that if you're doing something bash specific you're most likely doing something that would be done better using Python/Perl/Ruby, etc
Preferably the complexity in a shell script should be mostly handled by the programs invoked in it. All it has to provide is variables, basic control flow, subshells, and maybe some globbing.
It's a bit too late for that. Bash is pretty much the de facto default shell for linux. Telling people not to use bash scripts is like telling web devs not to use javascript or OS devs not to use C.
It may change in the future, but bash is so entrenched, it's going to take an extraordinary use case for people to move from bash.
https://linux.die.net/man/1/checkbashisms https://mywiki.wooledge.org/Bashism https://wiki.ubuntu.com/DashAsBinSh https://en.wiktionary.org/wiki/bashism
Oh, yes it is. Every important distribution uses bash as the default.
You don't know your history. Exactly such a move has already happened once, a decade ago. And it was a success.
* https://lists.debian.org/debian-release/2007/07/msg00027.htm...
* http://initscripts-ng.alioth.debian.org/soc2006-bootsystem/b...
* https://wiki.debian.org/BootProcessSpeedup#Using_a_faster_sy...
Some Linux distro don't ship it either (void, alpine IIRC).
And, most of the time, bash is used because people don't know any better, see e.g. https://github.com/OpenRA/OpenRA/pull/8405/files
I love finding and removing all the GNU/Linux-ism in my bash code when I move it from a dev host to my personal laptop. macOS coreutils don't have GNU longopts, surprise!
The installed binaries will have their names prefixed with `g`; e.g. `gdate` instead of `date`.
If something you want isn't part of coreutils, you can often install it directly; e.g. `brew install gawk` or the weirdly-named `brew install gnu-sed` (which names its installed binary `gsed`... go figure.)
Wow, this comparison is totally whack. Javascript and C are both standardized and the former is the only option when using a web browser.
It's never too late. We managed to get people off of ANSI C and there's a similar amount of difference between ANSI* C and C89 as there is between bash and sh.
* correction: pre-ANSI
This is nonsense. I am very familiar with web browsers. Javascript is standardized as ECMAScript:
https://www.ecma-international.org/publications/standards/Ec...
The "standard library" browsers implement is also standardized:
You're off your rocker.
>Did I say it was too late? Of course I didn't. Maybe if you took a step back from your fanboyism and read...
Here is a direct quote from your comment:
>It's a bit too late for that. Bash is pretty much the de facto default shell for linux.
>ANSI C and C89 are the same thing. ANSI C is just another name for C89... Sigh...
Derp, I meant K&R C or pre-ANSI C. My mistake.
Come on Drew, you know better than to do this here, regardless of how wrong someone is. If senior users set this kind of example how can we tell new ones not to?
But I wasn't wrong. I was right. But you wouldn't know that because your a gender studies major and not a CS major.
> If senior users set this kind of example how can we tell new ones not to?
And yet no flagging and no ban. How hypocritical of you. The guy instigated it and yet you banned the victim rather than the victimizer. Also senior? I've been using hacker news since the very beginning. Long before trash like you arrived daniel.
If you don't want to be banned, you're welcome to email [email protected] and give us reason to believe that you'll follow the rules in the future.
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3...
> Unfortunately, even in 2008, where shells without any function support are far and few between, there are pitfalls to avoid when making use of them. Also, finding a Bourne shell that accepts shell functions is not trivial, even though there is almost always one on interesting porting targets.
When a system does not (correctly) implement portable standards, it is a bug in the system and software should not be corrected to accomodate for it. autotools disagrees with me on this point.
Whelp, wrong right off the bat.
I'm gonna get down my my knees here and beg everyone reading: use POSIX shell. Do not write scripts with bash. Do not write scripts with zsh. Do not write scripts with fish. Use POSIX shell.
sh has a really bad interactive mode (so does bash), so I'm not gonna give anyone a hard time for using another shell as their day-to-day interactive shell. But, for the love of god, write your scripts with POSIX sh.