Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by MayeulC on Thursday March 16, 2023 @06:46PM (#35189974)

I agree that this is something missing on classic UNIX shells: typed output streams. I had this discussion a while back on HN, though I can't find it ATM (I wish there was a comment history search function). I am far from the first one who thought of that, and there are a few implementations of this idea.

Searching for that comment, I came across relevant stories:

https://news.ycombinator.com/item?id=27535357

https://news.ycombinator.com/item?id=25225113

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by gregjor on Friday March 17, 2023 @06:04AM (#35195107)

Typed output streams aren’t missing from Unix. The authors of Unix clearly describe, repeatedly, that Unix tools use plain text as the common format, and that everything should look like a file.

You’re right that standard Unix tools don’t have a concept of types in streams, but that decision got made deliberately. Types and formats got left as output details.

Analogously my Macbook Air doesn’t have a fan, by design, not by accidental omission.

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by MayeulC on Friday March 17, 2023 @02:42PM (#35201132)

Right, that was an unfortunate choice of words. It's not missing, but leads to (IMO) a proliferation of under-specified text-based data and stream exchange formats.

Having human-readable text as the lowest common denominator is a laudable goal. Shell scripting would however probably be improved if most tools offered alternative typed streams, or something similar. I am not convinced Powershell's approach is the best, but their approach is at least interesting.

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by gregjor on Saturday March 18, 2023 @04:34AM (#35207327)

Picking nits, but the decision to use plain text as the common data format in Unix does not rule out structured text (e.g. CSV, TSV, XML). Nor does it imply “human-readable.”

The original decision was about not proliferating specialized or proprietary binary formats, which was more of a norm back in the ‘70s than today. The goal was to make small single-purpose tools that communicated through a common interface (files) in a standard format (plain text). Unix succeeded and continues to succeed at that.

Nothing about those design decisions precludes tools using binary formats under Unix — image processing, for example. It just precludes using standard text-oriented tools on those formats.

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by mdaniel on Friday March 17, 2023 @03:58PM (#35202056)

Algolia supports searching comments, and while I don't know the "facet" syntax to pin it to your username, thankfully it's a pretty distinct word and https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... coughed up https://news.ycombinator.com/item?id=26797583 from 2 years ago. Is that the one?

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by MayeulC on Monday March 20, 2023 @03:23AM (#35228312)

Ah, interesting feature I didn't know about, thank you.

It's one of the parents in that comment chain (top comment on the story): https://news.ycombinator.com/item?id=26791597

[ Parent | on HN ]