News for hackers. Stuff that matters.
Re:Shell Style Guide by jon-wood on Tuesday May 15, 2018 @09:16AM (#17073653)
I'm genuinely curious, why? If you're writing generic scripts designed to run across multiple platforms then fine, but in the context of Google (or most other companies) they'll have a standard set of tooling available on all their machines, which presumably includes Bash if they're making that statement.

What about POSIX shell makes it superior for scripting?

Re:Shell Style Guide by simias on Tuesday May 15, 2018 @09:25AM (#17073706)
I don't feel as strongly about it as Sir_Cmpwn does but I agree with him. The logic goes something like: you should only use shell scripts for small scripts and not complex applications. Ergo the added functionality of bash scripts should not be necessary anyway and might actually lead you to believe that it's a proper programming language.

Or to put it the other way around: if you find yourself writing a POSIX shell script and find that you'd like to use more advanced features then one of two things can happen:

- You need portability to SunOS 4, so you suck it up and keep hacking your script because bash is not an option anyway;

- You don't need portability, so then why not use a proper programming language like python, perl or ruby rather than upgrading from crappy sh to slightly-less-crappy bash?

Re:Shell Style Guide by avar on Tuesday May 15, 2018 @09:48AM (#17073868)
This logic is insane. This is like a company that mainly does C++ or Java saying that if you must use a scripting language it must be Tcl, not Perl or Python, least you get too comfortable with it.

If you want a policy against shellscripts growing out of hand have a policy against that, not that you must use the lowest common denominator even though you have Bash everywhere.

And I'm saying this as someone who almost exclusively writes POSIX sh for portability reasons, but my software isn't meant to target just Google's infra.

Re:Shell Style Guide by simias on Tuesday May 15, 2018 @10:30AM (#17074143)
I think it would be perfectly sane for a company to dictate which scripting language their environment will support. Shell scripts are a bit different though because if you write a large enough software system on a unix-like OS you'll probably want to write a few shell scripts here and there to glue things together, so forbidding them completely would be a bit silly and counterproductive. Saying "avoid shell scripts, if you really have to write one it must be simple and use POSIX sh" is not particularly insane to me.

But anyway my argument was more about my opinion about a developer talking for myself, not a company style guide. Life's too short to learn two shell dialects so I can either learn POSIX which will work basically everywhere and will let me write any simple script just fine or write Bash whose added complexity I probably won't need or even want.

The fact that I'm an embedded dev probably biases my view though, I generally target environments where bash isn't an option anyway.

Re:Shell Style Guide by Sir_Cmpwn on Tuesday May 15, 2018 @09:24AM (#17073703)
Google also makes it difficult to compile Chromium on platforms where /usr/bin/python is python 3, because their shebangs are:

    #!/usr/bin/env python
Rather than the more portable:

    #!/usr/bin/env python3
Notably this causes problems for people on Arch Linux, and in the near future, Fedora. This post is presented as a style guide that others should adopt. Google may have bash ubiquiotously available internally, but their approach should not be taken as a model for others. Google's approach has caused real problems for their software's portability in exchange for dubious value-adds.

>What about POSIX shell makes it superior for scripting?

To address this specifically, I have written a blog article on the subject:

https://drewdevault.com/2018/02/05/Introduction-to-POSIX-she...

The main points are:

- bash is far from as ubiquitous as some people seem to think it is

- sh is formally standardized and has multiple competing implementations, whereas bash is defined by its implementation and there is only one

- sh is perfectly competent and easy to target, bash's value add is questionable

- By the time you need the things bash offers you might as well not use a shell script at all

Re:Shell Style Guide by mort96 on Tuesday May 15, 2018 @10:04AM (#17073970)
If you find yourself compiling Google-stuff a lot on Arch, you might find this useful: https://github.com/mortie/nixConf/blob/master/bin/py2

I wrote that script exactly because of the issue you described, and I got tired of changing what /usr/bin/python is symlinked to constantly. With that script, I just run `py2 gn gen out/Debug` or whatever, and my /usr/bin/python stays intact so everything else which expects python3 works, and only the `gn` process and all subprocess sees python2.

This definitely doesn't excuse Google's behavior, but I don't expect it to be fixed soon. My bug report about it (https://bugs.chromium.org/p/webrtc/issues/detail?id=7376) was closed as wontfix (though I'm not entirely sure the person who closed it understood that I was asking for the python scripts to explicitly use python 2, not for them to port the scripts to python 3).

Re:Shell Style Guide by moviuro on Tuesday May 15, 2018 @10:09AM (#17074012)
Clearly the WONTFIX was wrongly issued. The issue should be re-opened.
Re:Shell Style Guide by chrisfinazzo on Tuesday May 15, 2018 @10:24AM (#17074099)
The python v python3 thing is their (Py's implementers) own stupid fault; although the argument could be made that they didn't have any choice.

People have been on 2.7 for so long that a breaking change might be the only thing to get them to move over.

As Chromium is open source, I'm surprised they haven't really explained how people outside of the Googleplex should handle this.

Seems like they're left to their own devices.

Re:Shell Style Guide by hyperpape on Tuesday May 15, 2018 @01:39PM (#17075875)
I believe your claim that

  /bin/sh 
is standardized is incorrect, though I don't know how many actual systems fail to have it present. Source: https://news.ycombinator.com/item?id=17069408 (I checked the 2018 version of the spec, and it says the same).
Re:Shell Style Guide by Sir_Cmpwn on Tuesday May 15, 2018 @01:50PM (#17076016)
Aye, I was incorrect. I have pushed a correction, but it will take a moment to appear on the site.
Re:Shell Style Guide by joshuamorton on Tuesday May 15, 2018 @10:42AM (#17074246)
The second thing is not more portable than the first (unfortunately).
Re:Shell Style Guide by Sir_Cmpwn on Tuesday May 15, 2018 @10:51AM (#17074334)
Re:Shell Style Guide by joshuamorton on Tuesday May 15, 2018 @10:59AM (#17074402)
That, unfortunately, only works if you don't need to maintain backwards compatibility with older systems. `python` will exist and point to python2 on any system. `python2` may not if it predates or ignores the PEP. `python3` didn't even come installed by default on a lot of nixes until recently, making it the worst of option of the three.
Re:Shell Style Guide by Sir_Cmpwn on Tuesday May 15, 2018 @11:01AM (#17074428)
Of course, it will take time for this to become effective. But this is the portable solution. Systems for which this does not work are broken and should be fixed, software should not be updated to accomodate for broken systems.
Re:Shell Style Guide by joshuamorton on Tuesday May 15, 2018 @11:18AM (#17074569)
>software should not be updated to accomodate for broken systems.

Its not. The software was written, and maintains, backwards compatibility with systems that predate or partially predate the spec. The spec was written to be backwards compatible with those systems. Using the backwards compatibility option is correct if you wish to maintain backwards compatibility. Consider that Arch also technically violates Pep 394,

> for the time being, all distributions should ensure that python, if installed, refers to the same target as python2

We shouldn't update the shebangs in all of our software ever to accommodate a misbehaving, spec-non-compliant distro, should we?

Re:Shell Style Guide by wildmusings on Tuesday May 15, 2018 @09:32AM (#17073766)
Competing implementations sounds like a potential downside to me. That’s just more possible configurations where a bug could be hiding. Even if you use standard posix shell, you’d be wise to target a single implementation and stick to it.
Re:Shell Style Guide by _jal on Tuesday May 15, 2018 @09:48AM (#17073869)
This is exactly backwards. If you don't have separation of specification and application, you don't have a specification.

I mean, standards aren't necessary; interpreters can absolutely be snowflakes. But multiple implementations is a sign of strength, and intentionally targeting the bugs of one implementation is short sighted, if sometimes required.

Re:Shell Style Guide by Sir_Cmpwn on Tuesday May 15, 2018 @09:37AM (#17073791)
So you think that competition between gcc and clang is harmful to the C and C++ ecosystems? If you find bugs in an implementation of POSIX sh, you should report it to that implementation. Multiple competing implementations is a sign of a good standard, it proves the standard's correctness and demonstrates its maturity.

There is no standard to hold bash to. Any strange behavior of it might be decided as by design and kept forever (and replicated by anything which tries to be bash compatible) or it could be determined to be a bug and, when fixed, break the scripts which worked around it or depended on it. There's no way to know how this is going to shake out because there is no objective specification for the language. At least when there's a bug in one of these competing sh implementations that you're so worried about, it's objectively a bug and can be fixed.

Re:Shell Style Guide by wildmusings on Tuesday May 15, 2018 @09:50AM (#17073892)
To be more precise, my point was that you should only be targeting one of the implementations anyway. I don't see how it's a good use of people's time to be testing their scripts against every shell that claims to be sh-compatible, especially at a place like Google where the scripts are almost all being written to run on completely Google-controlled environments.

But I didn't think about the bug vs. by-design issue and how that is remedied by a well-defined standard. That does seem to be a factor in favor of using a standardized language. But it doesn’t seem different in principle than depending on any software. It is liable to change and leave you with a difficult decision of forking the library or modifying your code that consumes it. Given bash’s ubiquity, it seems reasonable to expect a good level of stability.

If you only target standards, the lowest-common-denominator of functionality, you will be poorer for it. Linux, for example, has all sorts of performance and functionality enhancing features over what POSIX provides.

Re:Shell Style Guide by Sir_Cmpwn on Tuesday May 15, 2018 @10:05AM (#17073976)
>If you only target standards, the lowest-common-denominator of functionality, you will be poorer for it.

In this case, you won't. I addressed this a few comments ago:

>By the time you need the things bash offers you might as well not use a shell script at all

Re:Shell Style Guide by jhasse on Tuesday May 15, 2018 @11:52AM (#17074882)
> competition between gcc and clang is harmful to the C and C++ ecosystems?

Yes. GCC will die eventually.

Re:Shell Style Guide by nurettin on Tuesday May 15, 2018 @09:25AM (#17073712)
Probably because he can't use bash code in his BusyBox based router
Re:Shell Style Guide by pas on Tuesday May 15, 2018 @11:23AM (#17074608)
And what's the magically amazing script that anyone would like to run on routers and AIXes alike, that it has to be POSIX portable? :)
Re:Shell Style Guide by JepZ on Tuesday May 15, 2018 @09:23AM (#17073695)
Maybe one day there will be another cool and dominating shell. That shell is more likely to be POSIX compliant than bash compliant.
Re:Shell Style Guide by oblio on Tuesday May 15, 2018 @09:32AM (#17073764)
All the cool new shells I've seen are either completely POSIX incompatible (powershell, xonsh, etc.) or they try to be bash supersets (zsh, osh, etc.).

Bash is a de facto modern standard.

The only new purely POSIX shells I know are for embedded stuff (dash) and that space is neither fast moving nor in dire need of a better shell, since most modern shell features target interactive use. Or they make coding easier and/or safer, in which case we're back to point 1, with the POSIX incompatibility.

Re:Shell Style Guide by JepZ on Tuesday May 15, 2018 @12:31PM (#17075197)
Maybe the missing POSIX compatibility is the reason why they haven't replaced bash yet.
Re:Shell Style Guide by oblio on Tuesday May 15, 2018 @03:59PM (#17077320)
Nothing will replace bash within my lifetime. Zsh didn't even though it was superior for over a decade.

Heck, Microsoft barely managed to supplant cmd 1-2 years ago, and that's in a non-standard, proprietary environment by what is arguably the biggest commercial platform maker in the world.