Shell Script Best Practices, from a decade of scripting things

Re:Shell Script Best Practices, from a decade of scripting t... by chriswarbo on Thursday October 27, 2022 @05:36AM (#33355265)

I agree with basically all of this. A few more:

Process-substitution can often be used to avoid intermediate files, e.g. `diff some-file <(some command)` rather than `some command > temp; diff some-file temp`

If you're making intermediate files, make a temp dir and `cd` into that

- Delete temp dirs using an exit trap (more reliable than e.g. putting it at the end of the script)

- It may be useful to copy `$PWD` into a variable before changing directory

Be aware of subshells and scope. For example, if we pipe into a loop, the loop is running in a sub-shell, and hence its state will be discarded afterwards:

  LINE_COUNT=0
  some command | while read -r X
  do
    # This runs in a sub-shell; it inherits the initial LINE_COUNT from the parent,
    # but any mutations are limited to the sub-shell, will be discarded
    (( LINE_COUNT++ ))
  done
  echo "$LINE_COUNT"  # This will echo '0', since the incremented version was discarded

Process-substitution can help with this, e.g.

  LINE_COUNT=0
  while read -r X
  do
    # This runs in the main shell; its increments will remain afterwards
    (( LINE_COUNT++ ))
  done < <(some command)
  echo "$LINE_COUNT"  # This will echo the number of lines outputted by 'some command'

[ Parent | on HN ]

Re:Shell Script Best Practices, from a decade of scripting t... by lhoursquentin on Thursday October 27, 2022 @05:42AM (#33355315)

What's interesting with this example of command1 | command2 is that some shells such as zsh will optimize the last member of the pipeline to be executed in the current process (nothing mandated by POSIX here), so effectively this works on zsh.

[ Parent | on HN ]

Re:Shell Script Best Practices, from a decade of scripting t... by karl42 on Thursday October 27, 2022 @05:41AM (#33355305)

> - It may be useful to copy `$PWD` into a variable before changing directory

Why not use pushd/popd instead?

[ Parent | on HN ]

Re:Shell Script Best Practices, from a decade of scripting t... by justsomehnguy on Thursday October 27, 2022 @05:55AM (#33355396)

a) if pushd fails you are doing things not in the target directory, and when you call popd you are now in a totally wrong place. set -o errexit should handle this, but there could be situations (at least theoretically) when you disable it or didn't enable it in the first place

b) you need to mentally keep the stack in your head when you write the script. And anyone else who would be reading your script. (Edit: including yourself a couple of months/years later)

c) pushd $script_invocation_path is easier to understand and remember.

Eg:

    $global:MainScriptRoot = $PSScriptRoot
    $global:configPath     = Join-Path $PSScriptRoot config
    $global:dataPath       = Join-Path $PSScriptRoot data

    $dirsToProcess = gci -Path $PSScriptRoot -Directory | ? Name -Match '\d+-\w+' | Sort-Object Name
    foreach ($thisDir in $dirsToProcess) {
        foreach ($thisFile in $moduleFiles) {
            . $thisFile.FullName
        }
    }

It's PowerShell, but the same idea. I use it in a couple of scripts, which call other scripts.

[ Parent | on HN ]

Re:Shell Script Best Practices, from a decade of scripting t... by chriswarbo on Thursday October 27, 2022 @07:44AM (#33356130)

pushd and popd are commands which change the current working directory. In contrast, variables like $PWD are expressions, which are far more flexible. For example, we can run commands like `diff "$OLD_PWD/foo" "$PWD/bar"` which reference multiple directories. Doing that with pushd/popd would be weird, e.g. `diff "$(popd; echo "$PWD/foo")" "$PWD/bar"`

[ Parent | on HN ]

Re:Shell Script Best Practices, from a decade of scripting t... by karl42 on Thursday October 27, 2022 @12:06PM (#33359656)

I fully agree. The parent wrote "before changing directory", so I assumed we are talking about the case where changing directory was necessary.

[ Parent | on HN ]

Re:Shell Script Best Practices, from a decade of scripting t... by IYasha on Thursday October 27, 2022 @05:48AM (#33355359)

I'd strongly recommend using /tmp/* temp dirs to save your SSD and speed up things.

[ Parent | on HN ]

Re:Shell Script Best Practices, from a decade of scripting t... by rascul on Thursday October 27, 2022 @08:25AM (#33356489)

> - It may be useful to copy `$PWD` into a variable before changing directory

$OLDPWD is set when you 'cd'. Also 'cd -' will take you back to the last directory.

https://pubs.opengroup.org/onlinepubs/9699919799/

[ Parent | on HN ]

Re:Shell Script Best Practices, from a decade of scripting t... by ilyt on Thursday October 27, 2022 @09:11AM (#33356953)

> The order of commandline args shouldn't matter.

Ugh I've seen so many (bash and non bash) cmdline tools that made it utterly annoying like that

The special place in hell goes to people who force users to write

    cmd help subcmd

instead of

    cmd subcmd --help

or ones that do not allow doing say

    cmd subcmd --verbose

because "verbose is global and doesn't belong to subcmd`

or ones where you need to write

    cmd --option1 subcmd --option2 subsubcomd --option3

and need to jump all over the cmdline if you want to add some option after previous invocation

and if you go "well but the option for command and subcommand might have same name" DONT NAME THEM THE SAME, that's just confusing people and search results.

[ Parent | on HN ]