News for hackers. Stuff that matters.

A comment on Demystifying Apache Arrow. Parent comment

Re:Demystifying Apache Arrow by tveita on Tuesday January 10, 2023 @07:04AM (#34323532)
Clickhouse or DuckDB are databases I would look at that support this use case pretty much "out of the box"

E.g. https://benchmark.clickhouse.com has some query times for a 100 million row dataset.

[ Parent | on HN ]
Re:Demystifying Apache Arrow by spaniard89277 on Tuesday January 10, 2023 @07:48AM (#34324033)
DuckDB is so simple to work with. It's only worth to look elsewhere with real big data, or where you really need a client-server setup.

I hope it receives more love.

[ Parent | on HN ]
Re:Demystifying Apache Arrow by IanCal on Tuesday January 10, 2023 @10:31AM (#34326310)
Duckdb is outrageously useful. Great on its own, but slots in perfectly reading and providing back arrow data frames, meaning you can seamlessly swap between tools when SQL for some parts and other tools better for others. Also very fast. I was able to throw away designs for multi machine setups as duckdb on its own was fast enough to not worry about anything else.
[ Parent | on HN ]
Re:Demystifying Apache Arrow by intelVISA on Tuesday January 10, 2023 @08:05AM (#34324217)
Having used all three I'd go with Clickhouse/DuckDB over Arrow every time.
[ Parent | on HN ]
Re:Demystifying Apache Arrow by sanderjd on Tuesday January 10, 2023 @08:38AM (#34324657)
Oh interesting - why?
[ Parent | on HN ]
Re:Demystifying Apache Arrow by intelVISA on Tuesday January 10, 2023 @08:47AM (#34324765)
They're easier to use and faster is the tl;dr.
[ Parent | on HN ]
Re:Demystifying Apache Arrow by nlittlepoole on Tuesday January 10, 2023 @10:12AM (#34326004)
100% agree.
[ Parent | on HN ]
Re:Demystifying Apache Arrow by lmeyerov on Tuesday January 10, 2023 @04:17PM (#34331641)
Probably for SQL (top n, ...), but not for wrangling & analytics & ML & ai & viz
[ Parent | on HN ]