News for hackers. Stuff that matters.
Re:My Laptop Is Faster Than Your Cloud: Announcing MotherDuck by danielmarkbruce on Tuesday November 15, 2022 @10:21PM (#33618410)
>> it already replaced pandas and dask for most of my common data eng tasks

This is interesting - why did you replace pandas/dask with this? What are you getting out of it?

Re:My Laptop Is Faster Than Your Cloud: Announcing MotherDuck by m_ke on Wednesday November 16, 2022 @05:07PM (#33631487)
It’s 100x faster, works on larger than memory data and let’s you run SQL over csvs, parquet, arrow tables and etc.

I mostly use it to extract data from a production DB, do basic preprocessing for ML training (which requires joins and filtering that gets ugly in pandas) and typical OLAP queries to compute statistics.

Re:My Laptop Is Faster Than Your Cloud: Announcing MotherDuck by danielmarkbruce on Thursday November 17, 2022 @09:53AM (#33639633)
Gotcha. Out of interest, have you ever done this with sqlite before? I do something similar with sqlite, but speed isn't a factor for me usually because the data is small.
Re:My Laptop Is Faster Than Your Cloud: Announcing MotherDuck by m_ke on Thursday November 17, 2022 @10:45PM (#33649903)
No because importing into sqlite can be pretty slow. With duckdb you can query parquet files directly.
Re:My Laptop Is Faster Than Your Cloud: Announcing MotherDuck by danielmarkbruce on Tuesday November 22, 2022 @01:54PM (#33709520)
Ok, I see that. Many thanks for the insight.