I replaced a ton of pandas code with an embedded DuckDB for internal data application and got a massive performance boost and (arguably) cleaner code.
Re:PRQL as a DuckDB Extension
by datadrivenangel on Thursday January 25, 2024 @11:43AM
(#39131505)
You can replace local postgres with DuckDB and it should be even faster for data analysis.
And if you want a severless hosted DuckDB with managed storage, sharing, UI, and ecosystem support, check out MotherDuck[0]!
(head of produck at MotherDuck)
Re:PRQL as a DuckDB Extension
by datadrivenangel on Thursday January 25, 2024 @03:56PM
(#39135158)
Do y'all have a reference architecture or templates for local development with CICD pipelines?
My last data engineering team struggled to get something like that working with BigQuery, so I'm super excited about the possibility of better data warehouse developer tooling.
Currently playing with postgress and dbt.