News for hackers. Stuff that matters.
Re:Use long flags when scripting (2013) by jwmhjwmh on Friday September 18, 2020 @12:54PM (#24519102)
If you're trying to stick to the POSIX standard, you have to use short options for standard commands like grep, since the long options are GNU extensions.
Re:Use long flags when scripting (2013) by dathinab on Friday September 18, 2020 @01:46PM (#24519715)
Does anyone care about the standard?

Let's be honest it's outdated, has a bad UX and is with the standards I would hold such a standard against today generally not so good.

Sure systems seen try to somewhat be POSIX compliance but from my experience not because they care about POSIX but because that happen to overlap with the idea to be somewhat compiland with other similar systems so that porting software/scripts is easier.

So if the systems you use support long options for POSIX commands go ahead and use them. Furthermore if the command you can are not in the standard anyway you again can use long options because they are as much standard as the short ones. Let's be honest this leaves very little use-cases where long options are a problem.

Re:Use long flags when scripting (2013) by ridiculous_fish on Friday September 18, 2020 @11:02PM (#24524568)
fish shell has been burned in the few places it strictly implements POSIX.

The most vivid example is test, aka /bin/[. POSIX specifies that, if test is invoked with one argument, you must "Exit true (0) if $1 is not null; otherwise, exit false" [1].

This means that `test --help` is forbidden from printing any help. It means that `test -d $argv` expands to `test -d` if $argv is empty, which then must "succeed" because "-d" is not null. It bites users over and over again [2].

Implementing this POSIX piece has resulted in more headaches, not fewer. I regret implementing a POSIX-conformant test, I wish I had just picked mostly-compatible but also-sane semantics.

1: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/t...

2: https://stackoverflow.com/questions/29635083/fish-shell-chec...

Re:Use long flags when scripting (2013) by a1369209993 on Saturday September 19, 2020 @02:41AM (#24525652)
> It means that `test -d $argv` expands to `test -d` if $argv is empty, which then must "succeed" because "-d" is not null. It bites users over and over again [2].

To be fair, while the POSIX behaviour is obviously wrong, using `$argv` on a bourne-style shell[0] is also wrong; that should be either `test -d "$argv"` or (if I recall the syntax correctly) `test -d "${argv[@]}"` if you actually intended argv to be a list rather than a single directory ("argv" suggests the former, but `test -d X` only accepts a single argument).

0: more specifically, a shell that does word splitting at the wrong place (after parameter expansion)

Re:Use long flags when scripting (2013) by kazinator on Friday September 18, 2020 @02:04PM (#24519941)
Last published in 2018 (and actively being worked on) is not outdated.

https://pubs.opengroup.org/onlinepubs/9699919799/

> Sure systems seen try to somewhat be POSIX compliance

The mainstream systems you know and use adhere very rigorously to POSIX. GNU Libc, Kernel, Coreutils, ... and their counterparts in BSD Unixes, proprietary Unixes, Cygwin and whatnot all take POSIX seriously.

POSIX is very helpful, and smart coders and sysadmins use it as one of their references for required behavior.

Re:Use long flags when scripting (2013) by chubot on Friday September 18, 2020 @08:08PM (#24523192)
Larry Wall said It's easier to port a shell than a shell script in 1998 ... 22 years later I still think that holds.

https://news.ycombinator.com/item?id=10104203

Although I would agree that there's a slight hole there: you still need to port stuff like coreutils and all the dependencies, which has been done of course. But I'd be happier if something like busybox was actually portable.

Re:Use long flags when scripting (2013) by kazinator on Saturday September 19, 2020 @04:19AM (#24526024)
I think it's easier to write a portable shell script today than 30 years ago.

The Autoconf system is predicated on the idea that writing a portable shells script is hard, and so we hide the shell programming behind a mountain of M4 macros.

It's not necessarily easier to port a shell script today than 22 years ago, which was not written with portability in mind. There are more shells with more extensions, and then beyond the language concerns, and the environments have exploded. A shell script can easily depend on all sorts of cruft you've never heard of. Oh, just install these five things from the following github repos ...

Re:Use long flags when scripting (2013) by samtheprogram on Friday September 18, 2020 @02:06PM (#24519970)
Yes, if you care about portability. Anything that's not Linux likely won't have GNU grep installed by default. This includes macOS and the BSDs.

If this is your own dotfiles or utilities, sure, but if I'm working on something collaborative, yes, I care about the standard.

Re:Use long flags when scripting (2013) by saagarjha on Friday September 18, 2020 @10:59PM (#24524551)
I stick to POSIX in my dotfiles precisely because I carry it around on many different systems and that’s the only thing that can keep it sane.
Re:Use long flags when scripting (2013) by samtheprogram on Saturday September 19, 2020 @02:58AM (#24525714)
Same here — I felt it would detract from the point I was trying to make so I didn’t mention it, especially since I don’t think most people run multiple OS’s :) At the least though, I would assume people regularly have collaborators that run different OS’s.
Re:Use long flags when scripting (2013) by dathinab on Sunday September 20, 2020 @10:05AM (#24534110)
If I would create something which needs that degree of portability I would never use a shell script.

It's just way too easy to run into a unexpected gotcha on some platform/configuration.

Re:Use long flags when scripting (2013) by chubot on Friday September 18, 2020 @08:05PM (#24523162)
Yeah I agree with this. I claimed a few years ago that you could view https://www.oilshell.org/ as a "better POSIX", and I still hold that view.

http://www.oilshell.org/blog/2018/01/28.html#limit-to-posix

Recent comment about this:

https://news.ycombinator.com/item?id=24428347

POSIX is just too limited and not what people use in practice.

If you want a portable shell script, in many cases my (biased) advice would be to make your script work on both bash and Oil. (Obviously there are short shell scripts which you may want to run on BSD, etc. This is more about big scripts, which POSIX falls down for.)

Oil already runs some of the biggest shell scripts in the world, many of them unmodified. Moreoever, when there's a patch necessary to run it, it often IMPROVES the program.

http://www.oilshell.org/blog/2020/06/release-0.8.pre6.html#p...

You'll be less tied to the vagaries of bash.

If anyone's script doesn't run under Oil, I'm interested. See https://github.com/oilshell/oil/wiki/What-Is-Expected-to-Run...

Re:Use long flags when scripting (2013) by nick238 on Friday September 18, 2020 @01:22PM (#24519435)
I've given up trying to be as pure and standard as possible. I could write all my scripts to use `sh`, but `bash` can make things far easier, and in my docker containers is often required by some other package anyway, so may as well just own it.
Re:Use long flags when scripting (2013) by stjohnswarts on Saturday September 19, 2020 @12:10AM (#24524961)
I agree, bash vs sh is night and day. Although these days I mostly use python. If I really really need something I can always shell out in python as well.
Re:Use long flags when scripting (2013) by efrecon on Friday September 18, 2020 @04:20PM (#24521472)
I usually test my POSIX scripts in an Alpine docker container. It has busybox by default, which only have a few flags and mostly short ones.
Re:Use long flags when scripting (2013) by Analemma_ on Friday September 18, 2020 @01:22PM (#24519444)
Unless you count things like Makefiles, I don't think I've ever written or encountered a "script" that was intended for use on multiple *nix flavors. This strikes me as a YAGNI situation: do it if it comes up, but not before.
Re:Use long flags when scripting (2013) by lhoursquentin on Friday September 18, 2020 @01:37PM (#24519623)
Yes most of the time the respecting POSIX to the letter is not needed, but it is of course satisfying knowing that your script can run fine on BSDs and other less common distributions :)

Though sometimes you don't need to go that far to break stuff, for instance switching from Fedora to Ubuntu. I've seen many scripts fail on debian derivatives because people think using #!/bin/sh as a shebang is fine since it works on their computer where sh was in fact a symlink to bash.

But on debian based distributions /bin/sh is often dash, not bash, and dash is basically the strict POSIX subset + local, all fancy stuff like [[ ]], &>, arrays, ... will fail.

Though this is less about long options here and more about general shell scripting.

Re:Use long flags when scripting (2013) by kazinator on Friday September 18, 2020 @02:14PM (#24520057)
You don't know that your script will run fine anywhere, if you've not actually run it there.

But, still, that's no reason to adopt a mindset of actively wrecking the chances of such success.

(Which is what passive ignorance amounts to, effectively).

Re:Use long flags when scripting (2013) by ori_b on Friday September 18, 2020 @01:46PM (#24519716)
This attitude is definitely something that makes my life harder. My OS is usually OpenBSD, and I constantly need to fix other people's scripts.

It's not difficult to do, but it is annoying. I don't blame people for ignorance, but it'd be nice if people thought about portability.

Non-portable scripts can also bite you on Linux, where Debian, for example, will swap out the shell from bash to dash for performance. but others will not. So, even within the Linux ecosystem, you can end up with scripts that behave differently across distros.

Re:Use long flags when scripting (2013) by rascul on Friday September 18, 2020 @07:50PM (#24523042)
> Non-portable scripts can also bite you on Linux, where Debian, for example, will swap out the shell from bash to dash for performance. but others will not. So, even within the Linux ecosystem, you can end up with scripts that behave differently across distros.

From the bash man page, "If bash is invoked with the name sh, it tries to mimic the startup behavior of historical versions of sh as closely as possible, while conforming to the POSIX standard as well." Also you should not be including bashisms if your shebang is sh and not bash. So using bash as sh shouldn't generally be a problem, if you stick to the published Shell Command Language [0], unless you hit one of the cases where the spec is a bit ambiguous and shells implement it differently [1].

[0] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... [1] https://stackoverflow.com/questions/16069339/different-pipel...

Re:Use long flags when scripting (2013) by ori_b on Saturday September 19, 2020 @11:48AM (#24528233)
> From the bash man page, "If bash is invoked with the name sh, it tries to mimic the startup behavior of historical versions of sh as closely as possible, while conforming to the POSIX standard as well."

From what I remember, it doesn't do a particularly good job.

Re:Use long flags when scripting (2013) by jjgreen on Friday September 18, 2020 @07:34PM (#24522910)
It's really useful to have a *BSD user or two in your userbase complaining about your non-POSIX linuxisms, it improves the portability of your code no end ...
Re:Use long flags when scripting (2013) by stjohnswarts on Saturday September 19, 2020 @12:12AM (#24524971)
I do 95% of my work on linux so bash it is. I'm not really concerned about other systems. I can always google later. It would take down my productivity quite a bit if I worried if all my scripting needed to be compatible with BSD or other unix varieties. YAGNI is mostly true.
Re:Use long flags when scripting (2013) by jfrunyon on Friday September 18, 2020 @03:19PM (#24520833)
Sounds to me like your choice of OS is what makes your life harder...
Re:Use long flags when scripting (2013) by ori_b on Friday September 18, 2020 @03:26PM (#24520904)
It makes my life easier in most ways. This is one of the costs.

And, again, this still bites whenever using different linux distros.

Re:Use long flags when scripting (2013) by jfrunyon on Friday September 18, 2020 @04:20PM (#24521476)
I never said that wasn't the case. That doesn't mean anyone else's attitude is the cause of problems caused by your own choices.

#!/bin/bash

Re:Use long flags when scripting (2013) by ori_b on Friday September 18, 2020 @04:27PM (#24521548)
I think you mean

    #!/usr/bin/env bash
Re:Use long flags when scripting (2013) by jfrunyon on Friday September 18, 2020 @04:37PM (#24521659)
Nope, I don't.
Re:Use long flags when scripting (2013) by kelnos on Friday September 18, 2020 @03:37PM (#24521037)
I think that's a reasonable attitude to have for your own personal scripts that you use on your own machines. But for stuff you do for work it might make sense to go for portability. Where I work most people develop on macOS, and our infra runs on Linux. I happen to develop on Linux, but if I'm writing development scripts that are intended to be run locally, I can't assume that GNU tools will be installed everywhere.
Re:Use long flags when scripting (2013) by kazinator on Friday September 18, 2020 @02:11PM (#24520027)
If you've ever run a ./configure script before building a program, you have encountered a script intended for use on multiple Unix flavors.
Re:Use long flags when scripting (2013) by samtheprogram on Friday September 18, 2020 @02:07PM (#24519982)
Portable shell scripts probably run on your system all the time, e.g. when installing development tools or packages, especially for cross-platform languages like Node.js, Ruby, Python, etc... If a hunk of software wants a shell script, it's gonna be portable.

Homebrew comes to mind, and any other piece of software that does the installation via a shell script.

Re:Use long flags when scripting (2013) by viraptor on Friday September 18, 2020 @06:34PM (#24522511)
There's a good chance you did. Lots of opensource developers use MacOS, so there's at least compatibility between that and Linux.