For the interactive general purpose data munging and quick execution of simple commands that the shell is best at, I really don't know what a better system would look like. It seems like a really hard problem. Anything purely text based ends up being fairly cumbersome to use for simple commands if it has to use real data structures (consider having to type (["a", "b"]) instead of a b to pass arguments with json style syntax or whatever). At least that was my experience of trying to write a very simple shell. There are a hell of a lot of people a hell of a lot smarter than me though.

It seems to me that a lot of shell scripts could be ported to other languages. Does DHCP on Linux need to use a shell script instead of python or something like that? The benefits of the shell grammar and semantics which are designed to make interactive use easy seem more like hindrances in a lot of those kinds of use cases. I assume it's largely done to make it easier for sysadmins to customise things. If I was a sysadmin I'd much rather learn python (and feel like I actually understood it) than the crazy byzantine grammar of bash. Maybe that's why I'm not a sysadmin.

This paper by Rob Pike might also be of interest: http://doc.cat-v.org/bell_labs/structural_regexps/se.pdf

>The current UNIX® text processing tools are weakened by the built-in concept of a line.

The reason the shell is used everywhere is because it's guaranteed to be installed (although DHCP used Bash explicitly), is much faster to start and run a simple script than Python, and its syntax is the command line that everybody should be familiar with.

I've hardly used it myself, but apparently Microsoft's PowerShell pipes typed objects between processes: http://technet.microsoft.com/en-us/library/dd347728.aspx

Actually, not between processes - all Powershell commands run in the same address space as the shell, and must be implemented in .Net. I don't think you can easily write an external process which takes a Powershell object directly as input.

Thanks for the correction, that explains so much--I was picturing a lot of DCOM craziness.