News for hackers. Stuff that matters.
Re:Filenames and Pathnames in Shell: How to Do It Correctly by js2 on Monday June 19, 2023 @08:50PM (#36398372)
This is an excellent resource. The only thing I'd add is: always use shellcheck too:

https://www.shellcheck.net/

(And I also like using shfmt on top of that.)

FWIW, I find writing POSIX-compliant shell rarely necessary. In my 25 years of writing shell scripts, they've nearly always been for automation on a specific OS, so portability was not ever a concern. These days I almost always use "#!/bin/bash" and only need to take care about the version of bash the system offers.

When I do have to write a script for more than one OS (e.g., macOS and Linux), I'll typically have bash on both systems, so I make use of bash'isms because they frankly make writing shell a lot more pleasant. I do need to take care however to use only those features supported by the older version of bash that ships with macOS.

Where I have to be more careful is in utilities like find, sed, etc which differ between BSD and Linux.

Re:Filenames and Pathnames in Shell: How to Do It Correctly by dreamcompiler on Monday June 19, 2023 @09:24PM (#36398621)
I write lots of scripts that need to work on both Linux and MacOS. Although both are POSIX, filenames in MacOS are case-insensitive and often contain spaces. This stuff plays havoc with bash scripts written by naive Linux programmers. Following TFA's recommendations ensures that your scripts will work on both.*

* well, except for the fact that MacOS defaults to zsh now.

Re:Filenames and Pathnames in Shell: How to Do It Correctly by js2 on Monday June 19, 2023 @09:37PM (#36398692)
To be clear: I agree with every recommendation in this article. I just find that writing POSIX-compliant shell is rarely needed.

I also write scripts for both Linux and macOS. They both provide bash, though macOS ships with an older version of it. So I use #!/bin/bash and stick to the older version that macOS provides. (Which often means ignoring the specific recommendations from shellcheck. e.g., macOS bash doesn't have readarray.)

What I have to be more careful of is differences in utilities like find, sed, etc.

Re:Filenames and Pathnames in Shell: How to Do It Correctly by ethanpil on Tuesday June 20, 2023 @12:30AM (#36399730)
Sorry to be so naive.. who is TFA?
Re:Filenames and Pathnames in Shell: How to Do It Correctly by js2 on Tuesday June 20, 2023 @12:52AM (#36399868)
The fine[1] article.

1. Originally from RTFM -> RTFA -> TFA, so feel free to use your own substitution for F, but in this case it's a fine article.

Re:Filenames and Pathnames in Shell: How to Do It Correctly by LanternLight83 on Tuesday June 20, 2023 @02:52AM (#36400584)
I've always thought of it as "the full article", having not made the connection :p

I think it's more broadly applicable

Re:Filenames and Pathnames in Shell: How to Do It Correctly by racked on Tuesday June 20, 2023 @09:32PM (#36413251)
The foregoing article
Re:Filenames and Pathnames in Shell: How to Do It Correctly by drbaba on Tuesday June 20, 2023 @02:41AM (#36400512)
> well, except for the fact that MacOS defaults to zsh now.

But Zsh is also POSIX sh compatible, is it not?

Re:Filenames and Pathnames in Shell: How to Do It Correctly by meinheld111 on Tuesday June 20, 2023 @03:50AM (#36400925)
No.

Afaik it can be thou

Re:Filenames and Pathnames in Shell: How to Do It Correctly by distcs on Tuesday June 20, 2023 @06:26AM (#36401996)
Zsh is not POSIX compatible. You need to enable many flags to make it POSIX compatible.

Zsh:

  $ A="foo bar baz"; for X in $A; do echo $X; done 
  foo bar baz
Dash:

  $ A="foo bar baz"; for X in $A; do echo $X; done 
  foo
  bar
  baz
Re:Filenames and Pathnames in Shell: How to Do It Correctly by Calzifer on Tuesday June 20, 2023 @06:55AM (#36402222)
> You need to enable many flags to make it POSIX compatible.

or use 'emulate sh' to enable many flags at once.

(and I personally would prefer if other shells would behave like Zsh and not word split unquoted variables by default)

Re:Filenames and Pathnames in Shell: How to Do It Correctly by JdeBP on Tuesday June 20, 2023 @07:28AM (#36402510)
This is true for the Bourne Again, Watanabe, and Korn shells too. The default mode is not the POSIX-conformant mode, but is the shell's native mode. This does not, however, make these shells "not POSIX compatible" any more than it makes the Z shell "not POSIX compatible". In all of them, one has to do non-default stuff, from setting particular environment variables through invoking the shell with a particular name to adding extra command-line options, to pick the POSIX-conformant mode.
Re:Filenames and Pathnames in Shell: How to Do It Correctly by JdeBP on Tuesday June 20, 2023 @07:15AM (#36402395)
The Z, Korn, Debian Almquist, Watanabe, and Bourne Again shells are all capable of running POSIX-conformant scripts. So if one has a POSIX-conformant shell script, one can use the Z shell to run it, yes.

The major factors when it comes to portability are, for all of them, that (a) the POSIX-conformant mode has to be specially invoked, and (b) even if invoked in conformance mode the non-POSIX parts are not always turned off. It's not that the shells won't handle POSIX-conformant scripts. They even mostly will in their (default) non-conformant modes. It's rather that it's difficult to discipline onesself to keep within the bounds of POSIX conformance when writing, even if one tests with conformance mode switched on.

The Debian Almquist and Watanabe shells are the best at preventing non-POSIX stuff creeping in by accident, because the former doesn't have much of it in the first place and the latter actually does things like turn off non-POSIX option arguments and builtins so that they generate errors in conformance mode and one can detect accidental "Yashisms" in testing. With the others, it's easier to accidentally use some of their various extensions and not spot it without explicitly testing with something in addition to the shell in its POSIX-conformant mode.

Re:Filenames and Pathnames in Shell: How to Do It Correctly by dreamcompiler on Monday June 19, 2023 @09:19PM (#36398583)
The author suggests using shellcheck in addition to his recommendations.
Re:Filenames and Pathnames in Shell: How to Do It Correctly by js2 on Monday June 19, 2023 @09:38PM (#36398700)
You're right! I totally missed that.
Re:Filenames and Pathnames in Shell: How to Do It Correctly by doctorpangloss on Tuesday June 20, 2023 @12:57AM (#36399899)
> In my 25 years of writing shell scripts, they've nearly always been for automation on a specific OS where portability is not a concern.

I've written hundreds of scripts for Busybox for Windows (https://github.com/rmyorston/busybox-w32), Linux bash and macOS zsh. POSIX compatibility means a lot.

WRT the specific concerns of arguments: if you are doing something complicated, use Python. ChatGPT writes it better anyway.

In my experience, more complex scripting is reinventing Gradle, Ansible, Terraform, YAML & similar.

Re:Filenames and Pathnames in Shell: How to Do It Correctly by js2 on Tuesday June 20, 2023 @02:00AM (#36400280)
I write a lot of Python too. I've written whole systems in it. Been using it since 1.5.2 days. And before that Perl. I know my way pretty well around sed and awk too and pretty much the entire Unix environment.

These are all just tools. They each have their strengths and weaknesses.

There are times when shell is the best tool for the job and when it is, I prefer to use bash.

If I have to write POSIX shell because it's a constrained environment, sure, I can do it. It's just been something that's rarely been a requirement.

Re:Filenames and Pathnames in Shell: How to Do It Correctly by coldtea on Tuesday June 20, 2023 @05:23AM (#36401568)
>if you are doing something complicated, use Python. ChatGPT writes it better anyway.

Amusing that this is even a factor

Re:Filenames and Pathnames in Shell: How to Do It Correctly by lelanthran on Tuesday June 20, 2023 @05:43AM (#36401711)
> WRT the specific concerns of arguments: if you are doing something complicated, use Python.

Introducing Python syntax for this reason alone is way over-engineering your script.

Yeah, if you're doing something that requires better data structures[1] then don't use the shell, but switching to Python just to call existing programs makes things more complicated, not less.

[1] Arrays, Trees, Dictionaries, etc

Re:Filenames and Pathnames in Shell: How to Do It Correctly by layer8 on Tuesday June 20, 2023 @05:32AM (#36401627)
> Arrays are really nice to have. One place I like to use them is for making long commands easier to read.

…at the cost of making it harder to grok for other people what that code is doing, and having them wonder if there’s some other important reason it has to be written that way.

Re:Filenames and Pathnames in Shell: How to Do It Correctly by latexr on Tuesday June 20, 2023 @06:18AM (#36401949)
> I also almost always prefer the self-documenting long-option to single-letter switches when writing shell scripts. Those single-letter options are to save you typing on the command-line. There's no reason to use them in a script.

Same applies to Stack Overflow answers and blog posts. If you’re teaching someone how to do something, always use the long flags so they can better understand it as they read. If you don’t know what the long option is, take the two seconds to check the manual.