I’m curious if you’ve tried the sh Python module [1]. In some ways it’s much prettier to read although the initial writing of it can be awkward to get used to at first.
I have not tried any of these libraries, though they look nice. The only times I've had to write scripts that make heavy use of subprocesses and pipes, I want them to work with only the standard library so I can just rsync them and they just work™.
from sh import mysqldump
from sh import gzip
gzip(mysqldump("example_db", _piped=True), _out="dump.sql.gz")
or if you don't want the magic import thing: import sh
sh.gzip(sh.mysqldump("example_db", _piped=True), _out="dump.sql.gz")
It's definitely foreign to shell scripting languages since the piping syntax is a bit different & you have to remember to do `_piped=True` since it's not parallel by default. But the default behavior is pipefail & exit on the command failing IIRC so it's more of a choose your poison thing (do you want a subtle perf issue or a subtle bug in your script not handling error states correctly). And I find it easier to read. + if you want to customize anything about gzip or not rely on needing the binary in the path, then you can just switch it to Python-native gzip pretty easily.That's my favorite feature. Conciseness if I'm just translating a script with progressive complexity options to migrate things that need more complexity or different requirements within the same script without having to rewrite it from scratch.
That's a clever way to go.
Sometimes the other way around works too: taking the most fiddly manipulations (say, date or time intervals) and re-homing them to a small Python script, but leaving the bash intact.
My biggest successes have been extracting these parts into very small stand-alone shell scripts which the Python application then invokes, which gives you the best of both worlds. The best example off the top of my head is `mysqldump | sed ... | gzip` with decent error handling (i.e. set -o pipefail in bash).
My environment isn't all that constrained, but I do have to deal with old Python versions (though at least I don't have to support 2.6 any more...), and a surprising amount of the features that would make replacing bash easier (subprocess.call especially) are only available fairly recently.