News for hackers. Stuff that matters.
Re:The Beauty of Unix Pipelines by ssivark on Thursday June 04, 2020 @05:57PM (#23422415)
Serialized bytestreams do compose better than graphical applications. But that is setting a very low bar. For example, allowing passing around dicts/maps/json (and possibly other data structures) would already be a massive improvement. You know what might be even better — passing around objects you could interact with by passing messages (gasp! cue, Alan Kay).

While posix and streams are nice (if you squint, files look like a poor man’s version of objects imposed on top of bytestreams), it’s about time we moved on to better things, and not lose sight of the bigger picture.

Re:The Beauty of Unix Pipelines by fao_ on Thursday June 04, 2020 @07:21PM (#23423179)
Yeah, but that makes every single file an object. Then you need plugins to deal with plain text objects, plugins to deal with every single type of object and their versions.

Objects aren't human readable and if they get corrupted the object's format is very difficult to recover, you need an intimate knowledge of both the specific format used (of which there might be thousands of variations). Plaintext, however, if that gets corrupted it's human-readable. The data might not be more compact (Although TSV files tend to come out equal), but it's more robust against those kinds of changes.

Have you ever examined a protobuf file without knowing what the protobuf structure is? I have, as part of various reverse-engineering I did. It's a nightmare, and without documentation (That's frequently out of date or nonexistent) it's almost impossible to figure out what the data actually is, and you never know if you've got it right. Even having part of the recovered data, I can't figure out what the rest of it is.

Re:The Beauty of Unix Pipelines by ssivark on Thursday June 04, 2020 @07:36PM (#23423299)
But that seems like just ignoring the problem, like an ostrich shoving one’s head into the sand!

> Have you ever examined a protobuf file without knowing what the protobuf structure is?

Are you referring to an ASCII serialization or a binary serialization?

In a typical scenario, your application needed structured data (hence protobuf), and then you serialized your protobuf into an ASCII bytestream... so how are you ever worse off with a protobuf compared to a text file, provided you make a good choice of serialization scheme? Clearly text files cannot carry enough structure, so we have to build more layers of scaffolding on top?

Ultimately files are serialized on to disk as bits, and “ASCII” and “posix” are just “plugins” to be used when reading bitstreams (serialized file objects). Plaintext is not readable if the bits get corrupted or byte boundaries/endianness gets shifted!

To have the same amount of of robustness with higher-order structures I imagine all one needs is a well-defined de/serialization protocol which would allow you to load new objects into memory? Am I missing something?

Re:The Beauty of Unix Pipelines by Firadeoclus on Friday June 05, 2020 @11:04AM (#23429370)
You could solve that problem with a system which mandated that structured data always come with a reference to a schema. (And where files as blobs disconnected from everything else only exist as an edge case for compatibility with other systems).
Re:The Beauty of Unix Pipelines by flukus on Thursday June 04, 2020 @08:14PM (#23423601)
> For example, allowing passing around dicts/maps/json (and possibly other data structures) would already be a massive improvement.

It is allowed, you can pass around data in whatever encoding you desire. Not many do though because text is so useful for humans.

> You know what might be even better — passing around objects you could interact with by passing messages (gasp! cue, Alan Kay)

That's a daemon/service and basic unix utils make it easy to create, a shell script reading from a FIFO file fits this definition of object, the message passing is writing to the file and the object is passed around with the filename. Unix is basically a fulfillment of Alan Kay's idea of OO.

Re:The Beauty of Unix Pipelines by boomlinde on Friday June 05, 2020 @05:51AM (#23426722)
> For example, allowing passing around dicts/maps/json (and possibly other data structures) would already be a massive improvement.

Well, it is allowed. You can of course serialize some data into a structured format and pass it over a byte stream in POSIX to this end, and I think this is appropriate in some cases in terms of user convenience. Then you can use tools like xpath or jq to select subfields or mangle.

> (if you squint, files look like a poor man’s version of objects imposed on top of bytestreams)

If all you have is a hammer...

Re:The Beauty of Unix Pipelines by cobalt on Thursday June 04, 2020 @08:17PM (#23423629)
try powershell, its on linux too
Re:The Beauty of Unix Pipelines by Koshkin on Thursday June 04, 2020 @09:08PM (#23423963)
Indeed, if it wasn’t for its reliance on .NET, it looks like PowerShell could be a massive improvement over Unix shell.
Re:The Beauty of Unix Pipelines by AnIdiotOnTheNet on Friday June 05, 2020 @10:48AM (#23429173)
There are projects like NuShell that are similar to PowerShell but don't rely on .NET.
Re:The Beauty of Unix Pipelines by majkinetor on Friday June 05, 2020 @01:31PM (#23431227)
There is nothing like PowerShell.
Re:The Beauty of Unix Pipelines by AnIdiotOnTheNet on Friday June 05, 2020 @01:43PM (#23431417)
I agree that all of them so far seem to be lacking in many ways compared to PowerShell, but saying they are "nothing like" PowerShell isn't a very accurate statement in my opinion.
Re:The Beauty of Unix Pipelines by majkinetor on Friday June 05, 2020 @01:53PM (#23431564)
No, that is totally accurate - there is nothing even close to PowerShell.

You were inaccurate on the other hand - what is similar to PowerShell ?

Re:The Beauty of Unix Pipelines by AnIdiotOnTheNet on Friday June 05, 2020 @04:00PM (#23433245)
Structured object piping, for one. What is this thread about again? Oh right, piping.
Re:The Beauty of Unix Pipelines by majkinetor on Saturday June 13, 2020 @04:22PM (#23512371)
"Structured object piping" is not a product.
Re:The Beauty of Unix Pipelines by AnIdiotOnTheNet on Monday June 15, 2020 @04:38PM (#23532690)
That's some non-sequitur argument you got there.
Re:The Beauty of Unix Pipelines by pjmlp on Friday June 05, 2020 @02:18AM (#23425587)
Some consider not relying on C a massive improvement.