Re:DuckDB – in-process SQL OLAP database management system
by syntaxfree on Friday February 10, 2023 @12:56PM
(#34743135)
It’s a drop in alternative to SQLite that’s column-oriented/OLAP. I’ve been profiling entire projects in production switching between SQLite and duckdb (no clear conclusions yet)
[ Parent
| on HN ]
Re:DuckDB – in-process SQL OLAP database management system
by nerdponx on Friday February 10, 2023 @01:11PM
(#34743366)
I suppose that leads to a broader question: when should you use an in-memory database, and when should you use a data frame library? The distinction between the two seems to be getting blurry (which maybe is a good thing).
Re:DuckDB – in-process SQL OLAP database management system
by IanCal on Friday February 10, 2023 @02:43PM
(#34744713)
Very blurry. The answer now is just "whichever is easier for the small part of the task right now". Since duckdb happily talks arrow, you can use pandas for part of it, quickly do some SQL where that is easier (with no data copying) then switch back to pandas for something. You don't really have to choose which one to use any more.