Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, and tabular JSON

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by avodonosov on Thursday March 16, 2023 @09:11AM (#35181452)

How is it better that SQL for these tasks on tabular data?

That's the first question I have after reading the title. Haven't read the article.

Edited the first sentence. Originally it was "Why it is not called SQL if works on tabular data?".

My point, if one wants cat, sort, sed, join on tabular data, SOL is exactly that. Awk is too powerful, not sure about it.

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by garciasn on Thursday March 16, 2023 @09:20AM (#35181560)

1. Because it doesn't use SQL syntax.

2. Because it's closer to an amalgamation of the standard shell scripting tools (cut, sort, jq, etc) than it is to a SQL variant.

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by kbouck on Friday March 17, 2023 @04:16AM (#35194519)

and miller also has a powerful (and intuitive imo.) dsl should you need to go beyond what the simple command line switches offer.

so, it remains simple and concise for the easy problems, yet can scale up to address the trickier ones as well.

i've used csvkit, sqlite, jq, etc. miller is my favorite tool for data-fu (with sqlite being a close 2nd)

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by LMMojo on Thursday March 16, 2023 @09:18AM (#35181533)

Because SQL (Structured Query Language) is the language used to access/manipulate the data, not the name for that kind of data. There have been, and are, many databases, which essentially use tabular data, which are not SQL database.

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by coldtea on Thursday March 16, 2023 @10:25AM (#35182392)

>How is it better that SQL for these tasks on tabular data?

It has far better handling of a CSV/TSV file on the command line directly and is compasable in shell pipelines.

>My point, if one wants cat, sort, sed, join on tabular data, SOL is exactly that.

SQL is a language for data in the form of tables in relational databases.

While it can do sorting or joining or some changes, it is meant for a different domain than these tools, which other constraints, other concerns, and other patterns of use...

You don't need to load anything to a db, for starters.

You also normally don't care for involving a DB in order to use in a shell script, or for quick shell exploration.

You also can't mix SQL and regular unix userland in a pipeline (well, with enough effort you can, but it's not something people do or need to do).

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by holy_diver on Thursday March 16, 2023 @11:02AM (#35182854)

DuckDB is actually pretty good at this kind of thing.

Doesn't need to load anything to DB

Can be used in shell

Can read from stdin and write to stdout

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by avodonosov on Thursday March 16, 2023 @11:34AM (#35183283)

csvsql is the first google result for "sql in command line for csv"

https://towardsdatascience.com/analyze-csvs-with-sql-in-comm...

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by coldtea on Thursday March 16, 2023 @01:51PM (#35185525)

Yes, I know. It's not that there isn't several ways to do it, it's that it's not really a good fit for the command line, except in the "I want to reuse SQL that I already know".

The problem isn't in having a way to use SQL to query the data from the command line, it's that SQL is long winded and with syntax not really fit in a traditional shell pipeline.

[ Parent | on HN ]

Re:Miller: Like Awk, sed, cut, join, and sort for CSV, TSV, ... by avodonosov on Thursday March 16, 2023 @07:42PM (#35190574)

I submitted a ticket: https://github.com/johnkerl/miller/issues/1235

[ Parent | on HN ]