News for hackers. Stuff that matters.
Re:DuckDB Internals: Why Is DuckDB Fast? (Part 1) by skeeter2020 on Friday June 19, 2026 @12:26PM (#48600296)
>> DuckDB is amazing for any sort of fast data analysis when the data is small enough that it can fit on your laptop

I agree, and the dirty (not so) secret big data providers like Snowflake try to hide: the majority of your work is not big data and WILL fit on your local machine. My last company was spending $2M/yr on contract with Snowflake, and another million between Fivetran and Matillion. Of the 1200 clients using analytics maybe 2 had enough data to warrant "infinite scalability" and a dozen wanted Snowflake because they already had corporate warehouses in Snowflake (they probably didn't need it either). Turns out the Extract and Load could be handled by bog-standard C# code and a bunch of SQL, while almost everyone was better off with a DuckDB database running locally, often in the browser. You've probably heard YAGNI before (You Ain't Gonna Need It) but it's even more likely with "Big Data". #SmallDataConvert

Re:DuckDB Internals: Why Is DuckDB Fast? (Part 1) by tomjakubowski on Friday June 19, 2026 @12:39PM (#48600432)
Folks have been beating this drum for as long as I've worked in software, dating to the Hadoop era, and it remains true today. So much of "big data" only appears big because it's poorly stored, or is represented wastefully (in persistent storage or in memory).
Re:DuckDB Internals: Why Is DuckDB Fast? (Part 1) by xnx on Saturday June 20, 2026 @10:33AM (#48609516)
SSD was a big turning point. Before that, there was more case for a database server.
Re:DuckDB Internals: Why Is DuckDB Fast? (Part 1) by nxm on Saturday June 20, 2026 @05:46AM (#48607862)
A good portion of users querying Snowflake at large companies are not technical so you can’t expect them to run DuckDB, not to mention data access controls
Re:DuckDB Internals: Why Is DuckDB Fast? (Part 1) by lanstin on Saturday June 20, 2026 @12:41PM (#48610623)
Greybeam (the company writing the blog) offers a service to proxy Snowflake and route queries in the fly to DuckDB or Snowflake based on predicted size. Saves a lot of money.