News for hackers. Stuff that matters.

A comment on Big data is dead (2023). Parent comment

Re:Big data is dead (2023) by WesolyKubeczek on Monday May 27, 2024 @07:55AM (#40489955)
> just go wild with CLI programs, I know dozens that work on CSV and XML

...or put it into SQLite for extra blazing fastness! No kidding.

[ Parent | on HN ]
Re:Big data is dead (2023) by pdimitar on Monday May 27, 2024 @07:57AM (#40489969)
That's included in CLI tools. Also duckdb and clickhouse-local are amazing.
[ Parent | on HN ]
Re:Big data is dead (2023) by WesolyKubeczek on Monday May 27, 2024 @08:01AM (#40489989)
I need to learn more about the latter for some log processing...
[ Parent | on HN ]
Re:Big data is dead (2023) by fijiaarone on Monday May 27, 2024 @08:46AM (#40490244)
Log files aren’t data. That’s your first problem. But that’s the only thing that most people have that generates more bytes than can fit on screen in a single spreadsheet.
[ Parent | on HN ]
Re:Big data is dead (2023) by thfuran on Monday May 27, 2024 @08:53AM (#40490295)
Of course they are. They just aren't always structured nicely.
[ Parent | on HN ]
Re:Big data is dead (2023) by WesolyKubeczek on Monday May 27, 2024 @09:18AM (#40490456)
Everything is data if you are brave enough.
[ Parent | on HN ]
Re:Big data is dead (2023) by c0brac0bra on Monday May 27, 2024 @08:48AM (#40490265)
clickhouse-local had been astonishingly fast for operating on many GB of local CSVs.

I had a heck of a time running the server locally before I discovered the CLI.

[ Parent | on HN ]