News for hackers. Stuff that matters.
Re:Bash 5.2 by pstuart on Tuesday September 27, 2022 @01:28PM (#32998393)
Improvements are always nice but portability matters, so depending on new features isn't possible until it could safely be assumed to be ubiquitous (cough, not you Apple).

Then there's the part where building anything of any significant complexity is not probably a good idea in a shell script.

Re:Bash 5.2 by chasil on Tuesday September 27, 2022 @01:36PM (#32998477)
If portability is your primary concern, then you want a POSIX shell, or at least to ensure that your bash is always in POSIX mode. There are lots of behavior differences when bash's POSIX mode is set.

Debian brought the Almquist shell in as /bin/sh, and is maintaining it with strict compliance to POSIX. This displaces bash as the system shell, but bash is still assigned as the interactive shell.

http://gondor.apana.org.au/~herbert/dash/

This is an older standard for the behavior of the POSIX shell. There are many common shell features that are not here (arrays, networking, coprocesses, fancy substitution, and much more). Doing without them increases portability.

https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...

Re:Bash 5.2 by rwmj on Tuesday September 27, 2022 @01:49PM (#32998674)
In my view Debian boiled an ocean for no discernable gain and caused a lot of trouble for everyone in the process. Fedora is using bash and I hope we continue rejecting the mistake Debian made.
Re:Bash 5.2 by stouset on Tuesday September 27, 2022 @04:04PM (#33000430)
If you want to write scripts to target bash, target bash with your shebang.

If your shebang is `/bin/sh`, then it's nice to have a strictly POSIX-compliant shell.

Re:Bash 5.2 by chasil on Tuesday September 27, 2022 @04:16PM (#33000574)
Bash does not "turn off" all its extra functionality when called as /bin/sh, it just alters the behaviors that are clearly in conflict with (what was) POSIX.2.

Arrays are still available in POSIX mode, even though they do not comply.

Re:Bash 5.2 by stouset on Wednesday September 28, 2022 @12:34AM (#33004283)
Which is why I think it’s good that Debian has a POSIX-strict `/bin/sh`. I have no idea what bash’s flawed POSIX mode has to do with this.
Re:Bash 5.2 by rwmj on Tuesday September 27, 2022 @04:55PM (#33000986)
Except if any program calls system(3) which always using /bin/sh. I maintain many core Linux packages and this Debian nonsense is a constant irritation with no discernible benefit.
Re:Bash 5.2 by tinus_hn on Tuesday September 27, 2022 @06:48PM (#33002133)
If it doesn’t work with ‘this Debian nonsense’, you’re doing it wrong and you’re contributing to a bad faux dependency on bash as /bin/sh.

The Unix world is better off if there is the option of using another shell that isn’t bug for bug compatible with bash.

This behavior is what leads to systems that have to emulate ancient bloated interfaces because they need to support applications that use apis that are defined as ‘how that program does it’. That’s bad. We should avoid it. Avoiding it is a benefit.

Re:Bash 5.2 by chasil on Tuesday September 27, 2022 @05:43PM (#33001516)
One secondary reason for the dash choice is speed.

The dash shell has been reported to be four times faster than bash.

That definitely impacts boot time. POSIX compliance is not the only benefit.

Re:Bash 5.2 by hnfong on Wednesday September 28, 2022 @10:12AM (#33008183)
system("/usr/bin/env bash -c '...'");
Re:Bash 5.2 by chasil on Tuesday September 27, 2022 @01:55PM (#32998755)
Debian dash is useful to keep my scripts honest.

Busybox actually takes dash, and then sprinkles a few bash/korn features back onto it (notibly, not arrays).

If you want a lot of people to use your scripts, getting them working in dash can help a great deal.

The POSIX mode in bash exists because bash itself predates POSIX by nearly a decade.

Re:Bash 5.2 by rwmj on Tuesday September 27, 2022 @04:56PM (#33000994)
What does "keep my scripts honest" even mean? Just use bash and don't make everyone else boil your ocean.

It's like saying you'll never use any Linux feature except those strictly defined by POSIX.1. Why would you do that?

Re:Bash 5.2 by chasil on Tuesday September 27, 2022 @05:33PM (#33001396)
Android uses mksh. If I write a script that uses read -p to present a prompt, it will fail on Android because it means coprocess on mksh.

The fundamental reality is that GPL does not run on iOS or Android (in userland). If you want to run scripts on those platforms (and they are ENORMOUS), then you cannot use bashisms.

Full stop.

Re:Bash 5.2 by kazinator on Tuesday September 27, 2022 @06:15PM (#33001819)
Note: BusyBox has two shell implementations in it, selected at compile time.
Re:Bash 5.2 by malkarouri on Tuesday September 27, 2022 @02:36PM (#32999260)
Are you building scripts for the Debian distro? Given that they already include bash and actually support it for interactive use, what would be the issue for you scripting using bash? I am sure that the experience of moving to dash wasn't fun at all. But users in general are not affected that much as far as I understand.
Re:Bash 5.2 by rwmj on Tuesday September 27, 2022 @04:54PM (#33000975)
I'm maintaining a bunch of core Linux packages which have had to make many adjustments over years for dealing with this Debian nonsense. I can see no justification for it since Fedora is using bash just fine and doesn't have to deal with this.
Re:Bash 5.2 by chasil on Tuesday September 27, 2022 @08:22PM (#33002884)
I definitely feel great sympathy for your position.
Re:Bash 5.2 by Ferret7446 on Saturday October 01, 2022 @01:35AM (#33044185)
>There are lots of behavior differences when bash's POSIX mode is set.

AFAIU correct POSIX scripts will behave correctly with Bash POSIX mode. Incorrect POSIX scripts will not behave incorrectly with Bash POSIX mode as one might expect with a more faithful shell implementation.

IMO Bash POSIX mode is not a problem for running scripts, although you should test your scripts using a strictly shell implementation if you will then distribute those scripts to other users/platforms.

Re:Bash 5.2 by harry8 on Tuesday September 27, 2022 @01:37PM (#32998496)
Serious question, why should I care about portability to Apple in shell scripts?

Why not make all your shell scripts #!/bin/bash and anything where that doesn't work is trying to be a second class citizen[1] in unix and so maybe let them? Or tell users to install Bash - which they can probably do if they're using shell scripts..?

[1] Or maybe deliberately trying to break portability by market power abuse.

I know I don't need tell anybody here that Apple is not their friend, merely a supplier who will maximize their revenue at your expense when it suits them.

Re:Bash 5.2 by yjftsjthsd-h on Tuesday September 27, 2022 @04:09PM (#33000483)
> Serious question, why should I care about portability to Apple in shell scripts?

It depends on your target audience, of course, but Darwin is one of the major current OSs and wanting to support it is reasonable.

> Why not make all your shell scripts #!/bin/bash and anything where that doesn't work is trying to be a second class citizen[1] in unix and so maybe let them? Or tell users to install Bash - which they can probably do if they're using shell scripts..?

Well, for starters AFAIK /bin/bash on Darwin will give you bash 3.2, which is a bit old and feature-poor. Of course you can (and perhaps even should) simply install a new version but then it won't be at /bin/bash, it'll be under /usr/local or /opt or whatever. And BASH isn't POSIX so I'd personally argue that trying to force it makes your stuff the "second class citizen in unix".

Honestly I'm not sure what position you're trying to argue. "Ignore the second biggest desktop OS because they don't ship the latest version of the non-standard scripting language I like"?

Re:Bash 5.2 by gogoyubari on Tuesday September 27, 2022 @04:32PM (#33000748)
>"Ignore the second biggest desktop OS because they don't ship the latest version of the non-standard scripting language I like"?

There's nothing wrong with that position. Bash is not a "non-standard scripting language people like", it is the de-facto standard on Linux (heck, it was the first program Torvalds ever run on Linux) and the most widespread implementation of POSIX shell in the world. If Apple chosed tivoization[1] over freedom, so I can choose the #!/bin/bash shebang.

[1] https://en.m.wikipedia.org/wiki/Tivoization

Re:Bash 5.2 by yjftsjthsd-h on Tuesday September 27, 2022 @06:47PM (#33002122)
> Bash is not a "non-standard scripting language people like", it is the de-facto standard on Linux (heck, it was the first program Torvalds ever run on Linux)

That sounds like the definition of not being standardized. BASH is the most common shell implementation on GNU systems (no, not Linux; Alpine is a Linux, and so is Android), but that doesn't make it a standard, only common. It's like claiming that nobody should care about anything but Chrome because it's "the de-facto standard".

> and the most widespread implementation of POSIX shell in the world.

BASH can run POSIX sh scripts, but POSIX sh can't run BASH scripts. If you're only using POSIX features, then it's not a problem, but if you're only using POSIX features we wouldn't be having this argument.

> If Apple chosed tivoization[1] over freedom, so I can choose the #!/bin/bash shebang.

Could I at least talk you into using `#!/usr/bin/env bash` so your scripts will work on a wider slice of the Linux universe? Even on Linux distros that exact path isn't a given (guix and nix send their regards), and you're completely breaking compatibility with the BSDs and illumos distros.

Re:Bash 5.2 by joombaga on Wednesday September 28, 2022 @12:29PM (#33009895)
> Could I at least talk you into using `#!/usr/bin/env bash` so your scripts will work on a wider slice of the Linux universe? Even on Linux distros that exact path isn't a given (guix and nix send their regards), and you're completely breaking compatibility with the BSDs and illumos distros.

You're also breaking compatibility with newer versions of bash on macOS that almost definitely wouldn't be installed at /bin/bash. I use 5.2 from brew.

Re:Bash 5.2 by tpoacher on Tuesday September 27, 2022 @05:11PM (#33001173)
bash isnt posix???

if bash isnt posix then wth is? if you told me "British English isnt actually English" I'd be less surprised than bash and posix.

Re:Bash 5.2 by yjftsjthsd-h on Tuesday September 27, 2022 @06:37PM (#33002024)
> bash isnt posix???

No

> if bash isnt posix then wth is?

POSIX sh (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...).

Re:Bash 5.2 by plonk on Wednesday September 28, 2022 @11:03AM (#33008814)
POSIX defines a shell that's not completely compatible with Bash. IIRC Ubuntu's `sh` command actually runs bash in a special "POSIX-compatible" mode that's really bare-bones.
Re:Bash 5.2 by chungy on Tuesday September 27, 2022 @02:44PM (#32999364)
> Why not make all your shell scripts #!/bin/bash

Because it's not always installed there. See, for example, FreeBSD where bash typically lives in /usr/local/bin/bash. Or in the Linux world, Nix and Guix.

Use "#!/usr/bin/env bash" to account for all these possibilities.

Re:Bash 5.2 by Ferret7446 on Saturday October 01, 2022 @01:38AM (#33044196)
If we're going to be pedantic, env doesn't have to be installed at /usr/bin/env either. See https://www.felesatra.moe/blog/2021/07/03/portable-bash-sheb...
Re:Bash 5.2 by tpoacher on Tuesday September 27, 2022 @05:06PM (#33001121)
i cant remember where I saw it now, or what the main argument was, but I remember reading a rather authoritative article recommending against the env approach, and recommending people stick to #!/bin/bash regardless of the above, otherwise valid point.

Dunno if anyone here with a better memory remembers the article to link here ...

Re:Bash 5.2 by asicsp on Wednesday September 28, 2022 @12:06AM (#33004138)
Re:Bash 5.2 by andrewmcwatters on Tuesday September 27, 2022 @01:52PM (#32998718)
You should care about POSIX Shell Command Language compatibility because targeting anything above that will eventually require you to do more, not less, work.

If you haven't run into this, you just haven't written enough for the shell yet.

Once you run into it, you'll not want to write anything specifically for Bash or Zsh anymore unless you know beyond a shadow of a doubt that it will never be run in any other shell.

Re:Bash 5.2 by dolni on Tuesday September 27, 2022 @02:57PM (#32999549)
Nearly all of the shell scripts ever written don't need to run on Linux, and BSD, and Mac, and/or whatever esoteric Unixes that are still out there.

If you are writing a shell script with the intention of running in many places, fine.

But if you do that, the POSIX compatibility will be the least of your worries.

Re:Bash 5.2 by chasil on Tuesday September 27, 2022 @01:41PM (#32998555)
A few "gotchas:"

I will commonly "alias p=printf" in my shell scripts. This is fine in Almquist and bash when called as /bin/sh, but if called as /bin/bash it fails, because bash only honors aliases in POSIX mode.

In bash, reading with a prompt can be done with "read -p prompt var" but this fails in Korn because it's used for coprocesses. This means that your shell script will not run on Android, because mksh is the system shell.

I could probably think of a few more with some effort, and especially a check of the man pages.

p.s. I didn't downvote you.

Re:Bash 5.2 by joombaga on Wednesday September 28, 2022 @12:52PM (#33010185)
> bash only honors aliases in POSIX mode.

Aliases certainly _work_ in bash (even when not in POSIX mode. What do you mean by "honors"?

Re:Bash 5.2 by chasil on Wednesday September 28, 2022 @04:59PM (#33013180)
Bash will not expand aliases within a shell script unless it is in POSIX mode.

When not in POSIX mode, aliases are only expanded for interactive use.

  $ echo '#!/bin/sh \n alias p=printf \n p %s\\\\n "hello world!"' > ok.sh
  $ chmod 755 ok.sh
  $ ./ok.sh
  hello world!

  $ echo '#!/bin/bash \n alias p=printf \n p %s\\\\n "hello world!"' > nope.sh
  $ chmod 755 nope.sh
  $ ./nope.sh 
  ./nope.sh: line 3: p: command not found
Re:Bash 5.2 by tannhaeuser on Tuesday September 27, 2022 @06:10PM (#33001774)
Wat? I suggest to leave such emotions out of your choice of programming languages and other decisions that should be dictated by rationality. Not adhering to POSIX shell syntax buys you very, very little, but can only create problems down the road. Bash isn't even the standard non-interactive shell on mainstream Linuxen (eg Debian uses dash), and zsh is not only used on Mac OS.

[1]: https://wiki.debian.org/Shell

Re:Bash 5.2 by coldpie on Tuesday September 27, 2022 @04:01PM (#33000387)
Serious answer, because I have users/customers who I care about that use that platform, and sometimes I use it myself. A world built on absolutes falls to pieces really quickly.
Re:Bash 5.2 by superkuh on Tuesday September 27, 2022 @02:12PM (#32998985)
It's okay. Portability depends on the behavior of the developers on average, not the rate of new features added to a language. Bash developers are generally conservative and not going to be using the newest backwards incompatible features for at least a decade. Whereas, say, Rust dev demographics are more bleeding edge types and when they get new features they use them immediately.

That's why bash scripts can be interpreted on any OS from 1995 to now and Rust can only be compiled on an OS which has updated it's rustc in the last 3 months.

Gotta hand it to Bash devs. They do a good job.

Re:Bash 5.2 by chungy on Tuesday September 27, 2022 @02:41PM (#32999332)
> That's why bash scripts can be interpreted on any OS from 1995 to now and Rust can only be compiled on an OS which has updated it's rustc in the last 3 months.

You can get the same effect by trying to run scripts with bash 5.x features on a version of bash from 1995.

Rust code since 1.0 has the same level of backwards compatibility that bash does.

Re:Bash 5.2 by superkuh on Tuesday September 27, 2022 @07:41PM (#33002584)
>Rust code since 1.0 has the same level of backwards compatibility that bash does.

That's true and exactly what I said. It's the rust developer demographic that lacks the consideration to write backwards compatible code.

Re:Bash 5.2 by chungy on Wednesday September 28, 2022 @12:00AM (#33004102)
I'm not sure the message is being conveyed properly -- looking up the GNU archive, it seems Bash 1.14 was the current release in 1995. Are people still making sure their scripts are compatible in Bash 1.14? Surely there's a subset of scripts that happen to work, but with numerous features that have been added since, there are plenty of scripts that won't.

Backwards compatibility isn't really about whether you can run code on _old versions_, but if you can run _old code_ on new versions. Scripts that were written with Bash 1.14 should work today (barring all the other external dependencies they may have...), but scripts that are written today won't necessarily work on 1.14.

That's the same as the Rust analogy: code written with Rust 1.0 should compile and run today, but code written today won't necessarily work on Rust 1.0 (albeit, there must be a subset of new code that would happen to work).

Re:Bash 5.2 by superkuh on Wednesday September 28, 2022 @08:51AM (#33007289)
>Backwards compatibility isn't really about whether you can run code on _old versions_, but if you can run _old code_ on new versions.

Ah, sorry, I guess I mean forwards compabitibility then. The ability to interpret/compile code written by devs today on machines with software from years ago (or months ago in rust's case).

Re:Bash 5.2 by rwmj on Tuesday September 27, 2022 @01:46PM (#32998623)
For several projects, we simply define bash as a required dependency. All the bash scripts use "#!/usr/bin/env bash". Not had any issues with portability between versions of bash, and basically every platform we care about has bash at least as an alternative if not the default.
Re:Bash 5.2 by gogoyubari on Tuesday September 27, 2022 @01:45PM (#32998604)
There are circumstances in which portability is not required. People have been writing bash before Apple picked it up and people keep writing bash despite Apple getting sent to GNUlag by GPLv3.
Re:Bash 5.2 by frou_dh on Tuesday September 27, 2022 @01:45PM (#32998613)
Shared shell-scripts are only one side of the coin. The other is interactive use (incl. one's .bash_profile etc), where it doesn't matter what anyone else's system does or doesn't have installed.
Re:Bash 5.2 by BiteCode_dev on Tuesday September 27, 2022 @01:36PM (#32998482)
Portability for shell scripts is over rated in 2022.

If your shell scripts are small, it's easy to write several versions of them.

It they are long, you should use something else than shell scripts for your program, like python.

Worse case scenario, provision the same interpretter everywhere, vendor it, or use a compiled language.

There are some edge cases where portability makes sense (e.g: you are scripting for an heterogeneous parc of routers), but they are niche usages most companies don't care about.

Re:Bash 5.2 by mburee on Tuesday September 27, 2022 @01:40PM (#32998547)
Because pulling in a huge python installation on small/embedded systems is fun, just because some people are unable to keep their scripts POSIX compliant
Re:Bash 5.2 by BiteCode_dev on Tuesday September 27, 2022 @03:09PM (#32999735)
Scripting in bash in a portable manner on embedded system is exactly the niche use cases I talked about. The router example is aimed at it, because it's the right size ratio to be small, but have an OS running.

Still, the vast majority of IT projects do not fall in that category.

It exists, but it's a very, very small % of why you need a bash script. And in fact, even those advocating for bash portability almost never fall into the category of people that have to do it in their daily job.

I'll go even further and even state that a lot of embedded system don't even have bash scripting capabilities in the first place.

Re:Bash 5.2 by ms4720 on Tuesday September 27, 2022 @01:47PM (#32998629)
Tcl, lua, or a lisp/scheme work well here
Re:Bash 5.2 by chasil on Tuesday September 27, 2022 @02:16PM (#32999028)
The nice thing is that many, many shells implement POSIX, so there is enormous portability.

Busybox bundles the Almquist shell, and there is a Windows port available. This is the easiest and least intrusive way to run shell scripts on Windows.

Busybox advertises that it bundles bash, but this is not true - it's Almquist with some added bashisms.

It also bundles their own awk implementation, but not Tcl/lua/lisp/scheme.

Re:Bash 5.2 by kaba0 on Tuesday September 27, 2022 @01:50PM (#32998690)
It’s not the compatibility which is the problem, but the 3 bugs/line that bash causes.
Re:Bash 5.2 by Alupis on Tuesday September 27, 2022 @02:39PM (#32999308)
If you cannot take the time to learn the language, what makes you so confident your python doesn't also have 3 bugs per line as well?
Re:Bash 5.2 by kaba0 on Tuesday September 27, 2022 @03:22PM (#32999880)
Did you run shellcheck on it, to actually recognize all the errors, before you assumed it is correct?
Re:Bash 5.2 by Alupis on Tuesday September 27, 2022 @04:22PM (#33000628)
Writing bash scripts is really not that difficult - despite what folks might want to convince you of.

ShellCheck is a great tool, yes, and can be run natively inside your IDE. Just like any language, you need to learn the language and learn it's tools.

Blanket statements like "write all your scripts in python because there's less bugs" really just means the commenter is more familiar with python. Do enough script writing and you'll realize how absurd that statement is.

It's a lot like the folks that scream nobody should ever write a single line of C or C++ because it's "dangerous" - yet untold number of lines of C/C++ are written every day. A pencil can be dangerous in the wrong hands...

Re:Bash 5.2 by Arnavion on Tuesday September 27, 2022 @05:51PM (#33001582)
Yes I did. And I hope you ran mypy on yours.

BTW both shellcheck and mypy are equally optional, so I'm not sure what your high horse is about.

Re:Bash 5.2 by BiteCode_dev on Tuesday September 27, 2022 @03:11PM (#32999761)
Well I use both extensively, and I'm very confident that I can write a script of 1000 lines with much less bug and much better error handling that in bash, in half the time it would take me.

In fact, as soon as you need to use arrays, it's game over.

Re:Bash 5.2 by RealAlexClay on Tuesday September 27, 2022 @03:28PM (#32999960)
1000 lines in python is a bash oneliner and planning for failure is bad design.

>In fact, as soon as you need to use arrays, it's game over.

Why?

PS. comfortable with both. I don't draw lines based on LOC or arrays.