News for hackers. Stuff that matters.
Re:Miller – tool for querying, shaping, reformatting data in... by jasonpeacock on Wednesday December 22, 2021 @12:44PM (#29652625)
It's interesting watching these types of tools get re-invented periodically:

https://github.com/benbernard/RecordStream

It shows the unix model of many small, composable tools is very powerful, but also shows that POSIX is missing some essential pieces that everyone keeps trying to add/reinvent.

Re:Miller – tool for querying, shaping, reformatting data in... by dima55 on Wednesday December 22, 2021 @01:09PM (#29652952)
Lots of them out there. Here's one that repurposes the extends the standard unix tools to know about comments and column labels: https://github.com/dkogan/vnlog/
Re:Miller – tool for querying, shaping, reformatting data in... by pdimitar on Wednesday December 22, 2021 @01:41PM (#29653322)
If have any more links, please post them! I am a fan of collecting such tools.
Re:Miller – tool for querying, shaping, reformatting data in... by eatonphil on Wednesday December 22, 2021 @03:04PM (#29654309)
I just published dsq [0] for running SQL queries against CSV/JSON/Excel/Parquet/etc or just converting those files to JSON.

[0] https://github.com/multiprocessio/datastation/tree/main/runn...

Re:Miller – tool for querying, shaping, reformatting data in... by loudmax on Wednesday December 22, 2021 @01:09PM (#29652956)
Pretty sure JSON isn't part of POSIX.
Re:Miller – tool for querying, shaping, reformatting data in... by jasonpeacock on Friday December 24, 2021 @05:32PM (#29678408)
Correct, but the point is that POSIX is missing something - maybe a standardized text format? These tools all first convert from various formats into a single, queryable format, then build further tools on top of that queryable format.