News for hackers. Stuff that matters.
Re:Pure Sh Bible by cduzz on Sunday April 30, 2023 @04:03PM (#35766109)
I have to take exception to some of this; it's "technically correct" but like much of shell, likely full of terrifying edge cases that are best avoided.

For instance, using eval to have variables with variable names is madness.

$ var="world"

$ eval "hello_$var=value"

$ eval printf '%s\n' "\$hello_$var"

I suppose the fancy business with printf is flexing, but I suspect the majority of people scanning documents like this would prefer to have examples with echo "${variable%%string_modifier}"

Escape sequences are mostly dependent on you being on a VT102 compatible terminal; I'm not sure what happens if you're on an ASR33 and you send this to it...

I'd consider type checking (is this a float) to be similarly cursed witchcraft -- after all I've run into all sorts of "floats" that aren't just 123.456 strings...

Overall -- there's a huge range of things shell can do that you probably should just avoid. Similarly, even if you can do it in a clever shell way (the bit shifting math for instance, or extensive string manipulation) should likely be done with external tools just because people understand "oh, awk or sed -- I know this!" instead of "what the hell is this line noise?" If you're writing performance optimized shell, well, probably put the keyboard down and walk away for a bit and reconsider your life choices.

Good doc; I'll probably stick to the man page though.

Re:Pure Sh Bible by Linux-Fan on Sunday April 30, 2023 @04:40PM (#35766406)
> I suppose the fancy business with printf is flexing, but I suspect the majority of people scanning documents like this would prefer to have examples with echo "${variable%%string_modifier}"

POSIX itself contains some interesting notes about `echo` and `printf`, e.g. from the `printf` page:

"The printf utility was added to provide functionality that has historically been provided by echo. However, due to irreconcilable differences in the various versions of echo extant, the version has few special features, leaving those to this new printf utility, which is based on one in the Ninth Edition system."

The behaviour of `echo` is especially inconsistent for escape sequences. Often, `echo -e` is used to enable escape sequences for it but this is not POSIX compliant.

For any "standards-perferring" shell scripting pages I'd thus think `printf` is the right way to present it if only to advertise that this builtin exists.

> should likely be done with external tools just because people understand "oh, awk or sed -- I know this!" instead of "what the hell is this line noise?"

I am not actually sure about this "I know this" part. I think many users of AWK only ever use it to do some sort of "access the n-th column of input" as in `echo a b c | awk '{ print $2 }'`. Similar for `sed` which is often only known and used for "replace string a by string b in input" use cases.

> If you're writing performance optimized shell, well, probably put the keyboard down

There is some difference between performance optimized and "hugely inefficient" that can make the difference between a task taking minutes vs. a few seconds. It may not seem much but shell scripts are often integrated into automated processes (think of build environments or system startup tasks) and there, seconds quickly accumulate. Whether this is relevant to your use case, only you can know.

I found this "Pure Sh Bible" to be highly interesting, but there are some places where it has some rough edges and it may not show the "best practices" for standard use cases. Still a very useful resource.

Re:Pure Sh Bible by palunon on Sunday April 30, 2023 @05:25PM (#35766771)
> Similar for `sed` which is often only known and used for "replace string a by string b in input" use cases.

Sed has at least the vi users who may know ex command from the vi command mode. Awk needs to be learned for itself.

Re:Pure Sh Bible by cduzz on Sunday April 30, 2023 @06:27PM (#35767132)
There are a huge variety of things you can do in shell that you simply shouldn't. If you're adventuring into a world where it matters if there's a newline at the end of your output as a result of your script running on noodlebsd or ache or freeGum, you've already lost.

Don't play the game. Don't do things where it matters what the exact output of echo gives you.

Did you know that you can't put a null into a shell variable? You can't. Also, you shouldn't ever be in a position to care. Did you know that you can fiddle with the "IFS" to let you parse CSVs? Don't ever do that; it works but nobody will ever understand what the hell you did. You can make an case statement and evaluate it with eval to make your own shell script that writes its own shell scripts. Please don't.

All of these adventures are possible, and work fine, and I've done them and come back to the code a decade later and I both understand what I was trying to do and the code still works across a variety of bourne shell interpreters. Nevertheless, these are things that shouldn't be done except to flex.

Re:Pure Sh Bible by ndsipa_pomu on Sunday April 30, 2023 @04:59PM (#35766573)
It's well worth learning to use printf instead of echo to avoid all the various ways that echo can break or is not well defined.

https://mywiki.wooledge.org/BashPitfalls#echo_.24foo

Re:Pure Sh Bible by koolba on Sunday April 30, 2023 @09:21PM (#35768162)
The %q format alone justifies learning about printf. Vanilla echo is for chumps.
Re:Pure Sh Bible by hddqsb on Tuesday May 02, 2023 @09:24AM (#35787061)
Note that this is supported by bash/zsh/ksh but not by POSIX sh.
Re:Pure Sh Bible by hddqsb on Tuesday May 02, 2023 @09:56AM (#35787450)
Yes. printf is not a "flex", it's the only safe way to display arbitrary strings. The link mentions one good reason to avoid echo (string might be "-n"), but another important reason is that the behaviour of echo is implementation-defined if the string contains backslashes (e.g. `echo 'foo\nbar'` displays one line in bash but two lines in dash and zsh).
Re:Pure Sh Bible by chasil on Sunday April 30, 2023 @05:07PM (#35766632)
Here are my scary evals to use basic ANSI/vt220 color.

  N="$(printf '\033[')" x=30

  for a in Bl R G Y B M C W # 4-bit Black Red Green Yellow Blue Magenta Cyan White
  do eval $a='$N'"'"$((   x))"m'" \
         b$a='$N'"'"$((60+x))"m'" \
      ${a}bg='$N'"'"$((10+x))"m'" \
     b${a}bg='$N'"'"$((70+x))"m'"        # bX=bright Xbg=background bXbg=brgt bgnd
     x=$((x+1))
  done

  N="${N}0m"

  printf "$Y${Gbg}I am yellow on green.$N\n"
I pasted this into my Android terminal, and it works as advertised.
Re:Pure Sh Bible by fsckboy on Sunday April 30, 2023 @08:29PM (#35767845)
I am curious yellow

on green is all that prints out? What's the rest of it for?

Re:Pure Sh Bible by chasil on Sunday April 30, 2023 @11:19PM (#35768754)
It's a general set of assignments that let you make any colors that you like.

That's only an example. Use them all, if you want.

Re:Pure Sh Bible by throwawaaarrgh on Sunday April 30, 2023 @06:52PM (#35767299)
Actually, eval, much like global variables (which are maligned by "real programmers" everywhere) actually works fine for a janky shell script.

Also, don't use echo, use printf. echo has different semantics and potential footguns depending on the platform.

Re:Pure Sh Bible by jethro_tell on Monday May 01, 2023 @12:27AM (#35769087)
Isn't the problem with eval that you are running whatever is in the var? Probably variable expansion but maybe not and that could be not ideal
Re:Pure Sh Bible by throwawaaarrgh on Monday May 01, 2023 @03:16AM (#35769928)
Programmers do that without realizing it all the time. There's like five classes of exploits that are just programmers assuming whatever variable they are interpolating is safe because they assume their language handles all security problems for them.
Re:Pure Sh Bible by 1letterunixname on Sunday April 30, 2023 @09:25PM (#35768199)
I'm scratching my head at most of it and there's large portions that are incomplete or trivial. bash is ubiquitous. If you're trying to develop an entire web framework to run on a Bourne shell from busybox, dash, or FreeBSD 4, there's probably a better way to do it. Use the tool you have, not a hypothetical one compatible with 1983. YAGNI.

Here's a common pattern for appending to PATH system-wide once:

    # /etc/profile.d/foo.sh

    DIR=/opt/foo/bin

    case ":$PATH:" in
      *":$DIR:"*) ;;
      *) PATH="$PATH:$DIR"; export PATH ;;
    esac

    unset DIR
If zsh or bash is available:

    # $1 var
    # $2 prepend this string to var
    # $3 separator (: default if omitted)
    # doesn't remove or prevent duplicates
    prepend() {
      eval "export $1=$2\${$1:+${3-:}\$$1}"
    }

    # $1 var
    # $2 append this string to var
    # $3 separator (: default if omitted)
    # doesn't remove or prevent duplicates
    append() {
      eval "export $1=\${$1:+\$$1${3-:}}$2"
    }
Re:Pure Sh Bible by hddqsb on Tuesday May 02, 2023 @09:47AM (#35787338)
Your "zsh/bash" code doesn't actually use any zsh/bash-only features, so it is compatible with POSIX sh. But you have a quoting bug: you should escape the `$` in `$2` and `${3...}` to protect from double-eval.

I'd personally try to minimise the use of eval, e.g.

    prepend() {
        local val
        eval "val=\$$1"
        export "$1=$2${val:+${3-:}$val}"
    }

    append() {
        local val
        eval "val=\$$1"
        export "$1=${val:+$val${3-:}}$2"
    }
Re:Pure Sh Bible by hddqsb on Tuesday May 02, 2023 @12:33PM (#35790164)
(`local` is not in POSIX, I know)
Re:Pure Sh Bible by singron on Sunday April 30, 2023 @10:24PM (#35768491)
In bash, you can use `declare -n`. E.g.

    var=world
    declare -n tmp="hello_$var"
    tmp=value
    printf '%s\n' "$hello_world" # value
I highly recommend this pattern if you want to emulate jagged 2d arrays or other nested data structures in bash for some reason.