News for hackers. Stuff that matters.
Re:The fastest tool for querying large JSON files is written... by wdroz on Friday April 15, 2022 @06:28AM (#31038541)
Why they didn't compare with Polars [0]? According to the h2oai benchmark [1], Polars is fast.

[0] -- https://www.pola.rs/ [1] -- https://h2oai.github.io/db-benchmark/

Re:The fastest tool for querying large JSON files is written... by dmoura on Monday April 18, 2022 @05:13AM (#31068728)
The initial idea was to focus on cmd line tools... I added pandas for comparison, as it is one of the most used libs to work with datasets. I will either remove Pandas from the equation or add Polars. By the way, I run some benchmarks and polars seems a bit faster than spyql for the aggregation challenge, but does not scale (loads everything into memory)
Re:The fastest tool for querying large JSON files is written... by Tarq0n on Friday April 15, 2022 @10:02PM (#31048495)
Polars is for tabular data, a different kind of task.
Re:The fastest tool for querying large JSON files is written... by wdroz on Saturday April 16, 2022 @04:24PM (#31055784)
But you can read json from polars (or from pandas first then polars) then you can do the map/reduce/filter like in the benchmark.