Because Bourne shell is not a Python-class language like Ruby or perhaps even Scheme or AWK are. Things that are easy in shell can be hard in Python and vice versa: awkward wrangling of subprocess.Popen in Python can be a single line of shell; simple tree munging with lxml in Python can easily be a hundred or more lines of wasteful xmlstarlet invocations in shell. Parallel processing, even without the heavy artillery of make or GNU parallel, is easy in shell but hard in Python. Structured data is hard in shell but easy in Python.

(Similarly, I wouldn’t use ML or Haskell to write a web scraper, and I wouldn’t use Python to write a compiler.)

Perhaps a better way to put it would be that sloppy concurrency is easy. This doesn’t sound all that good, but it’s easy enough that I find myself parallelizing one-off things in shell that would be such a hassle to do nonsequentially in Python (compared to the magnitude of the problem) that the option wouldn’t even surface in my brain. (If I rewrite the shell script in Python later, the rewritten version is often sequential.)

The shell error handling story for “run stream processing steps in lockstep” parallelism is much better than for “spawn a bunch of identical children” parallelism, though, and that’s often enough.

   #!/bin/bash -eux -o pipefail

   TMPDIR=$(mktemp -d tempdirXXXXX)
   trap “rm -rf $TMPDIR” SIGINT SIGTERM ERR EXIT
   (
      cd $TMPDIR
      foo | grep x > foo.out & 
      bar > bar.out & 
      baz $TMP | tee baz.out &
   )
   wait

   tar -cf - -C $TEMPDIR | zstd -6 -z - | pv | ssh server.com bash -c “cat - > out.tag.zstd”

    ... | ssh server.com dd of=out.tag.zstd

    $ shellcheck myscript
     
    Line 1:
    #!/bin/bash -eux -o pipefail
    ^-- SC2096 (error): On most OS, shebangs can only specify a single parameter.
     
    Line 4:
    trap "rm -rf $TMPDIR" SIGINT SIGTERM ERR EXIT
                 ^-- SC2064 (warning): Use single quotes, otherwise this expands now rather than when signalled.
     
    Line 6:
          cd $TMPDIR
             ^-- SC2086 (info): Double quote to prevent globbing and word splitting.

    Did you mean: (apply this, apply all SC2086)
          cd "$TMPDIR"
     
    Line 9:
          baz $TMP | tee baz.out &
              ^-- SC2086 (info): Double quote to prevent globbing and word splitting.

    Did you mean: (apply this, apply all SC2086)
          baz "$TMP" | tee baz.out &
     
    Line 13:
    tar -cf - -C $TEMPDIR | zstd -6 -z - | pv | ssh server.com bash -c "cat - > out.tag.zstd"
                 ^-- SC2086 (info): Double quote to prevent globbing and word splitting.
                 ^-- SC2153 (info): Possible misspelling: TEMPDIR may not be assigned. Did you mean TMPDIR?

    Did you mean: (apply this, apply all SC2086)
    tar -cf - -C "$TEMPDIR" | zstd -6 -z - | pv | ssh server.com bash -c "cat - > out.tag.zstd"