I found this problem as well when doing a direct port including pipes from bash to python. It was the first iteration into python and I was lacking time to do a proper port into python native.

The python code to do piping ends up as longwinded and the plumbing of pipes ends up a massive headache so I wrote tidycmd to overcome that issue https://github.com/laurieodgers/tidycmd

Don't see why it "doesn't scale well", the overhead is roughly constant: use the stdout of one program as the input of the next.

That's not as easy as

    cat something | grep "this" | cut -f 1 | sed -e 's/.../.../' 

You end up writing too much code, it's very verbose. Some times symbols are what you want. In fact the biggest progress in the growth of Math happened when they tossed out doing math with words and bought in symbols.

> That's not as easy as

> cat something | grep "this" | cut -f 1 | sed -e 's/.../.../'

Literally none of this is actually useful if you're already in Python:

    (
        line.split('\t')[0].replace(…, …)
        for line in open('something')
        if 'this' in line
    )

> You end up writing too much code

I can believe that if you're calling to external processes to perform operations which are pretty much trivial in the language.

Of course, if you plan to use one $language alone, you can do anything in that $language.

This question is specific to pipes.

> Of course, if you plan to use one $language alone, you can do anything in that $language.

That's missing what I'm noting though, which is that you don't need pipes anywhere near a shell script if you can simply do more of your work in-language. Case in point being that the entire pipeline you cited as an issue has no reason to exist outside of a shell or shell script.

That's because regexes are an another pain in Python, You have to write a lot of boiler plate exception handling code. Plus the same boiler plate code for file operations.

It doesn't feel like the language was designed for these tasks.

You forgot setting -e and -o pipefail. In case "something" is missing or can't be read for some other reason, your script will just continue happily without warning.

But oh, if you do set -o pipefail the grep will stop the whole pipeline when none of the lines matches "this". So you have to keep fiddling with ${PIPESTATUS[0]}. And none of pipefail or PIPESTATUS are really portable.

Not so simple after all.