"sh" for python is actually pretty cool

https://amoffat.github.io/sh/

Available as `python3-sh` in Ubuntu.

This seems friendlier out the box then subprocess. Any reason to use this vs subprocess?

The trouble is that bash is necessary to setup python. For instance, the lingua franca of Docker is bash (well sh, but you should obviously change that).

I've been resisting learning (any more) bash for a while now, but I think I'm going to have to. This looks like it could reduce some of my terror at the insanities of shell scripts (why does one even need two quote characters that do different things?)

Ruby has two kinds of quotes as well (interpolating and non-interpolating), but that must be borrowed from sh

I'd wager Ruby borrowed from Perl, which was the original borrower from sh.

Js has 3 quotes, of which 2 do the same, and the remaining one does two different things...

And that's another language I've been avoiding learning ;)

edit: minor bugfix in example code

A couple recommendations that make bash a much saner and avoid entire classes of problems:

First, whenever you are expanding a list of args, use "$@"! That exact four character sequence expands to the properly quoted positional params ("$1" "$2" ...) with any necessary escaping included so each param expands as a single word. Almost all of the problems you've probably heard about bash "not handling spaces properly" or otherwise having problems with whitespace or strange characters in filenames are fixed by using "$@". If you're using arrays/hashes, you can get the same effect using "${somearrayorhash[@]}" (quotes included, just like "$@"). Removing the quotes or using the tradition $* is almost always a bug.

Second, always use explicit quotes/brackets! Forget that they were ever optional. Using "$@" fixes most of the whitespace-in-filename problems; expanding your variables with explicit quotes fixes the rest. Assuming these:

    showargs() { 
        echo "$# args"
        for i in "$@" ; do
            echo "arg[${i}]"
        done
    }
    declare -- name="filename with spaces\\!.txt"
    declare -A h='([a]="b c" [foo]="'\''bar'\'' \"baz\" qu*x" )'

Instead of using the traditional shortcuts (which cause problems):

    showargs $name
    # 3 args
    # arg[filename]
    # arg[with]
    # arg[spaces\!.txt]

    showargs ${h[*]}   # or ${h[@]}
    # 7 args
    # arg[b]
    # arg[c]
    # arg['bar']
    # arg["baz"]
    # arg[qu*x]

Always using quotes/brackets simply does the right thing:

    showargs "${name}"
    # 1 args
    # arg[filename with spaces\!.txt]

    showargs "${h[@]}"
    # 2 args
    # arg[b c d e]
    # arg['bar' "baz" qu*x]

Bash still has it quirks and strange historical baggage, but in my experi4nce, using these two rules (and actually taking the time to read the bash(1) manpage...) changed writings shell scripts from an annoying mess of buggy arcane incantations into an actually sane(-ish) programming language.

One set of quotes (double quotes) allows for interpolation of commands/variables. Try:

  $ echo ‘pwd’
  $ echo “pwd”

You're right, but it's neither of those for command interpolation. Double quotes enable variable expansion, single quotes do not. Command interpolation is back quotes, which is a carryover from sh. The more modern bash way is as follows:

   $ echo $(pwd)

Incidentally: how does one type back quotes on an iPhone?!

Edited to add: shellcheck [0] will flag the back quote usage if you're writing a bash script instead of a sh script.

[0] https://www.shellcheck.net/

Longpress single-quote.

That ... lacks discoverability.

`Thank you`!

Like most iPhone tips. How was I to discover long press space bar for cursor placement?! I like it, but discoverability...

It was much better on the 3D Touch equipped models, press “through” the keyboard to get cursor placement, press “through” again while on a word to select the whole word.

ShellCheck always points out backticks as "legacy".

Thanks for the correction

If it’s just a few lines or some one-off thing, I understand the use of bash.

I start to get the feeling that if you feel the need for shell scripts, it might be wise to pause and wonder of this is really the right approach long-term. Especially if you feel the need to put it in git or something.

My experience is that there are often I need to put in some checks for safety and before I know it, you create a mess of grep awk cut sed and you wish you started out with python.

Are you really that much in a hurry or do you have the time to calmly spend a little bit more time to ‘do it right?’

Or you literally just need to run a sequence of commands to create some stuff in the filesystem without doing any text processing whatsoever, which is my use case for the three or four large-ish BASH scripts I've written professionally. Make was totally inappropriate in that case because I would have had to enumerate a lot of intermediate files and would have wound up with a parallelizable series of mini-scripts that would need to run serially to work correctly. And Python would ultimately turn into a DSL that looks almost exactly like BASH because the problem domain is "run a bunch of commands in sequence" which is what BASH is designed to do.

Yes, I think this is exactly the right kind of reason to use bash.

Is there an elegant way to do unix-like piping in python?

- use shell=True and subprocess module i.e., use sh for what it is good for (one-liners that run external commands) and use python to glue it together in a sane manner (consider shell as a DSL in this case). - you could use the plumbum module to embed commands in Python itself https://plumbum.readthedocs.io/en/latest/#piping - for Makefile-like functionality invoke/fab could be used (flexible configuration, composition, debugging of the commands) https://docs.fabfile.org/en/2.6/getting-started.html#addendu...