The fastest tool for querying large JSON files is written in Python (benchmark)

Re:The fastest tool for querying large JSON files is written... by flurly on Friday April 15, 2022 @09:48AM (#31040190)

I personally tend to put all my data for analysis into clickhouse (which is written in C++ FWIW). My side project actually is all about ingesting JSON data into clickhouse and making it super easy to query, visualize, monitor and alert on JSON data.

You certainly pay a computation cost everytime you want to scan the data, but I think the flexibility more than makes up for it. You can take a read about how and why I built GraphJSON here if you're interested https://www.graphjson.com/guides/about

[ Parent | on HN ]

Re:The fastest tool for querying large JSON files is written... by mritchie712 on Friday April 15, 2022 @11:10AM (#31041097)

This is awesome. Assuming you saw that 22.3 has a JSON column type?

https://clickhouse.com/blog/clickhouse-22-3-lts-released/

ps - we're built on Clickhouse at Luabase (https://luabase.com/) and hiring across all role, send me a note ([email protected]) if you're into web3+analytics.

[ Parent | on HN ]

Re:The fastest tool for querying large JSON files is written... by qoega on Friday April 15, 2022 @10:10AM (#31040453)

Have you tried ClickHouse for dataset above? Interesting if it is faster or not

[ Parent | on HN ]

Re:The fastest tool for querying large JSON files is written... by zX41ZdbW on Sunday April 17, 2022 @02:55PM (#31063248)

It is appeared that clickhouse-local is faster than all of the rest, with a huge margin: https://colab.research.google.com/drive/1pGmfpNdGyzE1KEXDnSS...

[ Parent | on HN ]

Re:The fastest tool for querying large JSON files is written... by dmoura on Monday April 18, 2022 @03:51AM (#31068356)

Yes, it is much faster, I will be updating the benchmark and reposting

[ Parent | on HN ]