News for hackers. Stuff that matters.
Re:Show HN: Checksum.sh verify every install script by dundarious on Friday October 28, 2022 @03:50PM (#33376812)
There are two big problems with the use of `echo $s` in bash/POSIX sh:

1. Never use echo to output untrusted content as the first argument

Let's say `s='-e 1\n2'`, then `echo $s` will output:

> 1

> 2

Instead of:

> -e 1\n2

Always use printf if you want to start output with untrusted content, e.g., `printf %s\\n "$s"`.

2. Never use unquoted variable expansion when trying to exactly reproduce contents of the variable

Similarly, unquoted variable expansion re-tokenizes the contents and will not preserve spaces appropriately. Say `s='"a<space><space>b"'` (where each <space> is a literal ' ', HN seems to be collapsing 2 spaces down to 1), then `echo $s` will output:

> "a<space>b"

Instead of:

> "a<space><space>b"

You can get the latter with `echo "$s"` but use `printf %s\\n "$s"` to fix both issues.

PS: If you fail to use quoted expansion with printf, for example like so, `printf %s\\n $s`, then you'll notice the problem right away, as it will effectively turn that into `for i in $s ; do printf %s\\n "$i" ; done`. That's actually a very useful feature of printf if you know to use it.

Edit: These problems exist for bash/POSIX sh at least. Perhaps you're using a shell that works differently, like zsh, because otherwise issue 2 would probably have led to some checksum fails for you already.

Re:Show HN: Checksum.sh verify every install script by googlryas on Friday October 28, 2022 @04:04PM (#33377054)
Great post, you are wise in the ways of the shell. Minutiae like this is exactly why I stop writing shell scripts the moment I start, and reach for python or some other sane language. But, I can't help but respect when I see masters of sh work their magic.
Re:Show HN: Checksum.sh verify every install script by dundarious on Friday October 28, 2022 @04:44PM (#33377686)
Honestly, 90% of problems with scripts are people forgetting to put double quotes around stuff. The other stuff doesn't come up that much, and once you write a few decent scripts, the other stuff is as easy as noticing someone wrote `open = True` in Python, not realizing they've redefined a builtin function, and the fix is just do `is_open = True`.

So just put double quotes around all your variable expansions unless you know you shouldn't -- 90% of scripts would be "fixed" with just that. And don't bother putting curly braces into the variable expansion unless you know you need to. People tend to think `echo ${s}` is somehow better than `echo $s` when it's exactly the same -- the curly braces are just a way to allow you to, e.g., write `"${s}_"` as distinct from `"${s_}"`. AFAIK in fish `${s}` is identical to `"$s"`, but that's a different kettle of sh.

Re:Show HN: Checksum.sh verify every install script by xelxebar on Saturday October 29, 2022 @04:31AM (#33382144)
Try enabling shellcheck linting in your editor! It would immediately warn about unquoted variable expansions and the like. For the vast majority of "gotchas" shellcheck will prod you in the right direction.

That said, to write good shell scripts requires actually learning the shell paradigm, instead of just trying to write "Python with Shell syntax".

Re:Show HN: Checksum.sh verify every install script by wolletd on Saturday October 29, 2022 @05:38AM (#33382472)
I think there's place for both. When I build software installers for our embedded systems, the "frontend" tools production uses are written in a "sane" language, but the actual installation of software and configuration of the systems is done in shell. Because for running a bunch of apt/tar/cp/ln/sed commands to configure a Linux machine, it is the sane language.
Re:Show HN: Checksum.sh verify every install script by rnhmjoj on Friday October 28, 2022 @04:04PM (#33377057)
For more caveats like this one I recommend reading: https://www.etalabs.net/sh_tricks.html
Re:Show HN: Checksum.sh verify every install script by gavinuhma on Friday October 28, 2022 @04:01PM (#33376999)
Re:Show HN: Checksum.sh verify every install script by dundarious on Friday October 28, 2022 @04:05PM (#33377085)
Missed the other `echo $s` piped into shasum. But I echo the sentiment of the another commenter that I'd rather rely on `shasum --check` to give the OK or not.
Re:Show HN: Checksum.sh verify every install script by gavinuhma on Friday October 28, 2022 @04:10PM (#33377180)
Got it. Thanks.

Re --check, I suppose the way to do that would be to download the file to disk, which --check requires as fair as I can tell. So I could download the file to disk, --check, and then remove it. I think most of these installs scripts are trying not to leave any artifacts around from install, other than the resulting binary.

Re:Show HN: Checksum.sh verify every install script by dundarious on Friday October 28, 2022 @04:17PM (#33377273)
You only need to create a temp file for the checksum file, not the downloaded contents. In the below example, no file exists on disk with the contents of `$s`.

> $ s='1<space><space>2'

> $ printf %s\\n "$s" | shasum -a 256 > tmp.sum

> $ printf %s\\n "$s" | shasum --check tmp.sum

> -: OK

So you can just `printf '%s<space><space>-\n' "$c" > tmp.sum` and check with `printf %s\\n "$s" | shasum --check --status tmp.sum || { echo "checksum failed" > &2 ; exit 1 ; }`

Having to create temp files is a wrinkle (could probably avoid it by using process substitution if you want to give up on POSIX sh), but so is writing bash scripts in general.

Re:Show HN: Checksum.sh verify every install script by gavinuhma on Friday October 28, 2022 @05:02PM (#33377874)
Solid! I couldn’t figure this out which I why I stopped using “—-check”. I’ll take a look
Re:Show HN: Checksum.sh verify every install script by gavinuhma on Friday October 28, 2022 @11:17PM (#33380658)
Pr for that https://github.com/gavinuhma/checksum.sh/pull/4

I ended up trying with process substitution so no tmp file.

It works. Trying to decide if it’s more difficult to read

Re:Show HN: Checksum.sh verify every install script by gavinuhma on Friday October 28, 2022 @03:55PM (#33376873)
This is awesome. Thank you! I've been through so many iterations but it's been fun to improve
Re:Show HN: Checksum.sh verify every install script by yjftsjthsd-h on Friday October 28, 2022 @05:16PM (#33378026)
If I may pile on with a general suggestion for people writing shell scripts: Use shellcheck. Always. It will catch these things automatically for you:)