That sounds interesting, but either programs deal with schema transforms, or we'll have to write a lot of that kind of logic in XSLT or jq (kind of the same stuff we do with cut, sed, awk, sort and grep for unstructured streams). Not sure it's that useful a feature unless the domain of the programs is narrow.

I think it could be useful - from PowerShell's perspective, all you need is to wrap your "classical" executables to somehow accept and emit .NET objects, and you can carry on from there. Importantly, no more writing parsers in the pipeline or in your scripts.

I could see a whole library of such wrappers being built for common executables (and new popular OSS CLI tools could provide their own wrapper along with the tool).

You can wrap your .NET, JVM, Python, Perl or C functionality in a small program that links to the library you want, that deserializes your XML or JSON, reads your command, and acts on the incoming object stream, spitting it to stdout in a neutral, preferably human readable, format that can then be piped into the next command.

Oh! Wait! This is what tools like cut, jq, grep, awk, and sed to.

And they won't tie you to a certain object model.

> Oh! Wait! This is what tools like cut, jq, grep, awk, and sed to.

Yes! Except, you have to use them to write one-shot parsers that are specific to both what you're piping from and what you're piping to! For n commands you may want to arbitrarily mix, that's n² parsers to write.

> And they won't tie you to a certain object model.

From the point of view of a system user, tying to a certain object model is a feature. It facilitates commands interacting well with no parsers in between. But even if you don't have a common object model, just wrapping the commands to produce structured output can already be helpful, because the parsers you'll be writing to glue commands will operate on structured, higher-level concepts. It's easier to map between two different representations of what a "file" or "process" is, than it is to manually dig them out of an unstructured text stream.

To use a stupid example, assuming hypothetical "kill-and-announce" that echoes a process name and sends a SIGKILL to its PID:

  ps aux | awk '{ ...extract columns... }' | sed 'rename columns' | xargs kill-and-announce ...

Would be harder to figure out than:

  wrapped-ps | wrapped-kill-and-announce

Or even with incompatible object models:

  wrapped-ps | select Target={id:$_.PID, name:$_.name} | wrapped-kill-and-announce

This is one of those eternal debates I think: on the one hand you have OOP and certain sorts of FP advocates advocating for extremely rich data models that eliminate certain classes of errors by design. On the other hand, you have the Unix notion of using a very generic data type (see also Perlis’s “rather 100 functions on one data structure than ten functions on ten data structures”) to maximize flexibility, at the cost of a certain amount of footguns.

Personally, I’m discovering more and more that CL’s REPL in SLIME is my preferred shell in part because the CL ecosystem has a rich array of operations on a small number of data-types (without forcing everything into binary streams), but also the ability to define ad-hoc datatypes as needed and a powerful system for manipulating them and integrating the custom data types into existing protocols.

Also, the claim that you need n^2 encoder/decoders isn’t actually true: conventions around data formats can reduce it to O(n) in the number of conventions.

> This is one of those eternal debates

Myself I know I'm stuck here. I've been thinking in circles for a while. At the moment, I'm leaning towards rich type system and structured data, but I'm not sure if it can be made fully convenient for the shell.

Do you know if anyone attempted to summarize this debate and distill some conclusions, or reframe it in a way that would allow to make progress?

> I’m discovering more and more that CL’s REPL in SLIME is my preferred shell

In terms of command line or scripting? SLIME is awesome, particularly with its interactive Inspector :). And I wrote my fair share of "scripts" as CL functions. But I can't seem to get used to using the REPL as a shell; I feel something is missing. Most likely convenience functions (that I didn't bother to write for myself yet).

That whole paragraph you wrote applies to another Lisp system, though: Emacs. In recent years, I've been leaning on it more and more for day-to-day operations. For instance, I rarely use shell for file operations - I mostly stick to dired. I end up scripting common operations straight in Emacs Lisp, and bind them to ad-hoc UIs (using the Hydra package). And of course utilize well-known packages like Magit (for Git). This recent experience makes me think that we could pull more common shell interactions and turn them from 1D to 2D interface. And I keep dreaming about merging it with PowerShell, in a way that allows Emacs to ingest and manipulate .NET objects nicely, while at the same time exposing its internals to PowerShell.

> Also, the claim that you need n^2 encoder/decoders isn’t actually true: conventions around data formats can reduce it to O(n) in the number of conventions.

I think this assumes conversions are lossless, which isn't the case in UNIX piping. When you invoke "tools like cut, jq, grep, awk, and sed", it's almost always to filter out most of the text in the pipeline.

To be honest, I’ve been thinking about a shell based on the JVM. Especially now with GraalVM, you can have truly polyglot programs running on top of it, with inlinig cross-language boundaries! But of course native code execution should be “bootstrapped” into that world somehow. (Actually, it can use LLVM IR and that as well can run a C function on js objects which is phenomenal, but people do want to run native software from a shell)

While you'll be able to invoke methods on returned objects, if you are piping stuff, you'll still need to deal with hammering the output of one program into the input of another.