News for hackers. Stuff that matters.
Re:Writing a Unix Shell by laumars on Monday May 29, 2017 @04:18PM (#14441190)
UNIX shells already optimise pipelines by running the commands along it concurrently. In your example, the first grep will be outputing it's results while it's still sucking in /var/log/somefile; and the second grep will be parsing the output of the first grep as and when the first grep outputs.

tail is a little different because it needs to find the end of the file but the greps certainly don't wait for each other to finish.

As for your "objects" point. The shell I'm working on (discussed elsewhere in this thread) uses JSON to pass objects around functions written for that shell while still falling back to standard flat text files when piping out to traditional UNIX/GNU/whatever tools. Though I still need to make the whole thing more intuitive.

Re:Writing a Unix Shell by eriknstr on Tuesday May 30, 2017 @06:05AM (#14444127)
>UNIX shells already optimise pipelines by running the commands along it concurrently.

I am aware of that. Still, performing multiple operations on the data within a single loop and with less copying would be an optimization.

Re:Writing a Unix Shell by laumars on Tuesday May 30, 2017 @06:38AM (#14444257)
I get what you're saying but technically those tools are running inside the same single loop (as that file is only iterated the once most of that pipeline) but distributed across multiple OS threads.

Shell scripts are surprisingly efficient at pipelined work where you're just dealing with streaming data (as your example mostly does). It's the more complex logic they fall significantly short on.

The weakest area and why I kept saying "most" of that pipeline would be the use of tail part way through the chain (as others have mentioned). But that could be optimised within Bash to something like:

    tail -r somefile | grep 'expression' | egrep 'expression' | head -n number | cut ...
(I think -r flag on tail is GNU rather than POSIX though)

You're object point is definitely relevant though. For example how many times have we seen shell scripts break because they don't consider spaces in filenames?

Talking about annoyances in Bash I'd also throw in exception handling as a major problem for traditional shells. Having Bash fail in expected ways and handle them cleanly is a huge pain in the arse.

Those above two points (and wanting cleaner syntax for iteration) are what drove me to write my own $SHELL so it does sound like I'm addressing some of your annoyances - albeit not all of them. However it's a long way from being something usable in production environments but I'm happy to access pull requests if you do ever feel you want to contribute to something that's trying to work towards at least some of your goals :)