News for hackers. Stuff that matters.
Re:The Rule of Silence (2006) by rtpg on Tuesday December 13, 2016 @07:15AM (#13166225)
Surely you can reconcile structured representations and something like the Unix command line.

Imagine if the default wasn't bash, but something like Ruby + pipes (or some other terse language).

What is the argument for shell scripts not working on typed objects? How much time has been lost, how many bugs have been created because every single interaction between shell scripts has to include its own parser. How many versions of " get file created timestamp from ls" do we need?

Something Windows does get right is the clipboard. You place a thing on the clipboard, and when you paste, the receiving program can decide on the best representation. This is why copy-pasting images has worked so magically.

I could see an alternative system where such a mechanism exists for shell programs.

Re:The Rule of Silence (2006) by pjungwir on Tuesday December 13, 2016 @01:29PM (#13169215)
Wow, the clipboard is really a thought-provoking comparison. I'm not sure if many people are quite aware of what you said, unless they've done desktop programming: when an application puts something on the clipboard, it can put multiple formats, so that when something else wants to retrieve it, it can use whichever format it prefers. This is how you get such good copy/paste interoperability between programs.

What if pipes worked the same way? What if we added stdout++, stderr++, and stdin++, and when you write to stdout/err++, you can say which format you're writing to, and you can write as many formats as you like. And then you can query stdin++ for which formats are available, and read whichever you like. And if stdin++ is empty, you could even automatically offer it with a single "text" format, that is just stdin(legacy).

The appeal of the Unix text-based approach is a kind of "worse is better". It is so simple and easy, compared to Powershell. The clipboard idea seems like it has a similarly low barrier to entry, and is even kind of backwards-compatible. It seems like something you could add gradually to existing tools, which would solve the marketplace-like chicken-and-egg problem.

You could even start to add new bash syntax, e.g. `structify my.log || filter ip_addr || sort response_size`. (Too bad that `||` already means something else....) Someone should write a thesis about this! :-)

Re:The Rule of Silence (2006) by buzzybee on Tuesday December 13, 2016 @04:54PM (#13171472)
Didn't the Amiga do something like this? (I'm not actually familiar with its OS, I've just seen allusions to how it handled file formats)
Re:The Rule of Silence (2006) by anexprogrammer on Wednesday December 14, 2016 @07:12PM (#13181046)
Amiga had datatypes.library, and also IFF filetypes (that were mostly lovely) designed be EA for DPaint. It's the way the world should work.

Let's say you're MS writing word for the Amiga.

They provide a datatypes description for doc files. this gives ability to read and write the format, and fingerprint it (not based on extension).

Now any program, old or new, that wants to read or write doc files can do. It's just there.

Re:The Rule of Silence (2006) by chei0iaV on Tuesday December 13, 2016 @07:55AM (#13166415)
The problem is that now simply providing user-friendly output is not enough. For every program or script you throw together you need to provide the text output for the user, and then the type object stream for piping. And then, the user would need to read documentation to see how to access each piece of data, what data type it is, what data type the other command takes, and maybe consider how to convert one to the other.

... at which point you basically have a scripting language, so you could just as well use an existing one (e.g. Ruby).

Re:The Rule of Silence (2006) by emodendroket on Tuesday December 13, 2016 @09:20AM (#13166917)
> The problem is that now simply providing user-friendly output is not enough. For every program or script you throw together you need to provide the text output for the user, and then the type object stream for piping.

In PowerShell if the result of a command is just an object the object is pretty-printed to the console in practice ends up looking pretty much like what a Unix command would have given you.

Re:The Rule of Silence (2006) by chei0iaV on Tuesday December 13, 2016 @10:49AM (#13167638)
With generic pretty-printing, your program output becomes generic.

Compare the output of "df -h" vs the PowerShell equivalent "gdr -psprovider filesystem", for example. One provides the data in dense (easy to follow) rows, while the other spaces it out across the whole screen, leaving large gaps of empty space around some columns while also cutting off data in others. The difference is especially noticeable of you have network shares with long paths.

PowerShell is probably nice for scripting, but I wouldn't want to have it as my shell.

Re:The Rule of Silence (2006) by emodendroket on Tuesday December 13, 2016 @11:02AM (#13167748)
You can pretty easily pass it to select and get just the properties you care about, or you could output to a different format with the various out- commands. I find it pretty good as a shell.
Re:The Rule of Silence (2006) by __jal on Tuesday December 13, 2016 @03:01PM (#13170365)
Options to select output? Sounds like advice from the Rule of Silence.

But.

- People above were just complaining about having to use sed to tweak output. I don't see why they would prefer a built-in filter to an external one. The external filter is far more flexible, and if that isn't enough, you can replace it.

- I'm generally not a fan of applications that tailor their output to what they think the human wants. Unless there is a deeply compelling reason, I want to see the same output on pts/0 as something down the pipeline. The reason for this is that picking up environmental hints to serve as implicit configuration is hacky, subject to error and and can later be the cause of really difficult-to-find bugs.

Perhaps I'm just irredeemably brainwashed. If you like a typed command line, Redmond has your back. For me, wanting types is a signal that I should start considering whatever little shell hack I'm working on complex enough to take it to a language that wasn't designed for ad hoc interactive use.

And at the same time, I really, very much do not want my command line to look like C#.

I get that a lot of folks these days are mostly GUI users who maybe type a little at git or run test suites from the command line, and not much else. I get why things like Powershell are appealing to such folks[1]. But when the command line is your primary interface, strong typing and OO hoop-jumping is huge waste of cognitive energy.

I do feel that Unix, to a first approximation, got the balance somewhere close to right. Loose coupling with lot of shared commonalities instead of a rigid type system and nonexplicit magic works really well for me, and if tighter coupling is a good idea, then I'll build it.

[1] Why they want to radically change the command line instead of using their favorite language to do systems stuff from the comfort of wherever they spend most of their time, I do not get.

Re:The Rule of Silence (2006) by emodendroket on Wednesday December 14, 2016 @03:00PM (#13179157)
I feel compelled to point out, re-reading this, that you've misunderstood. You pass the output to the Select command and indicate the object properties you want; it's not a feature that has to be built in into each command.
Re:The Rule of Silence (2006) by nailer on Wednesday December 14, 2016 @12:12PM (#13177582)
> People above were just complaining about having to use sed to tweak output. I don't see why they would prefer a built-in filter to an external one.

You don't see why someone would prefer 'select' to scraping with grep or awk?

Re:The Rule of Silence (2006) by emodendroket on Tuesday December 13, 2016 @03:38PM (#13170742)
Yeah, wow, all I have to do is write a bunch of regular expressions tailored to the unique output of this command instead of passing in a list of property names. It's so easy.
Re:The Rule of Silence (2006) by dahauns on Tuesday December 13, 2016 @06:26PM (#13172275)
To add to that: a list of property names you don't even have to exactly know beforehand, since the shell can easily deliver you the exact data structure and metadata of any object (cmdlets, parameters, results,...) usable on the shell.

I find the idea baffling that basically having to have the whole syntax tree memorized instead of the shell providing it for me is somehow less of a "waste of cognitive energy".

Re:The Rule of Silence (2006) by emodendroket on Tuesday December 13, 2016 @08:58PM (#13173189)
I really think people are mistaking "I'm not familiar with it" with "it's bad and poorly designed" because they've forgotten what it was like to first use the Unix command line.
Re:The Rule of Silence (2006) by nailer on Wednesday December 14, 2016 @12:19PM (#13177667)
Slightly more readable posh version:

    Get-PSDrive | Where Provider -like '*FileSystem'
Re:The Rule of Silence (2006) by michaelcampbell on Tuesday December 13, 2016 @09:35AM (#13167014)
> in practice ends up looking pretty much like what a Unix command would have given you.

So... no real improvement, then?

Re:The Rule of Silence (2006) by emodendroket on Tuesday December 13, 2016 @09:40AM (#13167057)
The improvement is that piping commands together doesn't require tedious text-munging while the console output looks basically the same. Which I think you could have figured out if you thought a little harder before jumping to snark.
Re:The Rule of Silence (2006) by romaniv on Tuesday December 13, 2016 @11:31AM (#13168059)
>What is the argument for shell scripts not working on typed objects?

Typed objects can make it harder to pipe commands together. How do you grep a tree when tree is an actual data structure and grep expects a list of items as input? You would need to have converters. Either specific converter between tree and list, or a generic one: tree->text->list.

>Something Windows does get right is the clipboard.

It useful, but the actual implementation is pretty bad. Opaque, prone to security issues, holds only single item, cannot be automated.

Re:The Rule of Silence (2006) by JadeNB on Tuesday December 13, 2016 @03:22PM (#13170579)
> Typed objects can make it harder to pipe commands together. How do you grep a tree when tree is an actual data structure and grep expects a list of items as input? You would need to have converters. Either specific converter between tree and list, or a generic one: tree->text->list.

To be fair, untyped objects also require converters, but at every boundary. That is, instead of having some pipes of the form `program -> mutually agreeable data structure -> program` and some pipes of the form `program -> unacceptable data structure -> parser -> program` (as happens with a typed language), you are guaranteed by a text-based interface always to have pipes of the form `program -> deparser -> text -> parser -> program`.

Re:The Rule of Silence (2006) by rtpg on Tuesday December 13, 2016 @09:51PM (#13173398)
grepping a tree would not be hard. You simply go down to the leaves and "grep" on the properties themselves. Inversely, structured data lets you easily get to a property.

For example, if you grep the output ls -l for a file named "1", you'll also get files with 1 in their timestamp. In text land, you have to edit the ls command to get simpler output. In structured land, you could edit your filter: ls -l | grep name~=1

You could imagine various structured data filter tools that could be built that wouldn't require modifying the input.

Though in this example, you can easily use awk to select the column, wouldn't it be nice to not have to worry about things like escape characters when parsing text?

Re:The Rule of Silence (2006) by patmcguire on Tuesday December 13, 2016 @06:17PM (#13172209)
How about protocol buffers and content negotiation? Your pipe figures out a proto that program a can emit and program b can consume.

ls -> repeated FileDescriptor files; | repeated string names;

Where FileDescriptor is whatever it needs to be, has all the info ls -l does. You have a heirarchy of outputs: if the next takes FileDescriptors, you give it FileDescriptors, if it doesn't you give it strings.

What would go to stdout goes through a toString filter.

Re:The Rule of Silence (2006) by Kalium on Tuesday December 13, 2016 @12:51PM (#13168802)
You are absolutely, completely correct. The two can certainly be reconciled!

There is one possible complication, though. The two would need to be reconciled in the same way by everyone who wants to write a shell tool. Given that even fairly simple standards (RSS, HTML, etc.) cause lots of failures to comply, what are the odds of near-universal compliance in a larger and more diverse ecosystem like shell utilities?

Re:The Rule of Silence (2006) by fulafel on Tuesday December 13, 2016 @09:26AM (#13166964)
Image processing works pretty well in shell pipelines because image formats are generally self-identifying.
Re:The Rule of Silence (2006) by cyphar on Tuesday December 13, 2016 @07:27AM (#13166285)
> What is the argument for shell scripts not working on typed objects? How much time has been lost, how many bugs have been created because every single interaction between shell scripts has to include its own parser. How many versions of " get file created timestamp from ls" do we need?

Aside: that's what the stat command is for. My big concern with types is how would you make sure that the output of a command will always have the right types? Otherwise you'll have runtime type errors which would be just as bad as runtime parsing errors.

Re:The Rule of Silence (2006) by junke on Tuesday December 13, 2016 @09:58AM (#13167186)
> Otherwise you'll have runtime type errors which would be just as bad as runtime parsing errors.

Parsing errors in script shells can easily go unnoticed (until you realize your data is corrupted).

Re:The Rule of Silence (2006) by tejtm on Tuesday December 13, 2016 @05:57PM (#13172048)
> How many versions of " get file created timestamp from ls" do we need?

none? 'ls' is for humans. 'stat' is for scripts.