News for hackers. Stuff that matters.
Re:Asynchronous I/O in DuckDB: Work, Thread, Work by pdet on Sunday August 16, 2026 @04:59AM (#49318232)
The main reason I decided to use a beefier machine is that it gives me flexibility when benchmarking, without the need to set up different environments. The CSV data, for example, is >80 GB. We can also “scale down the machine” for experiments where we want to stress-test lower-memory scenarios or use fewer threads by configuring DuckDB’s settings (e.g., SET memory_limit = '10GB'; or SET threads TO 1;).

(Disclaimer: I’m the author of the blog post.)

Re:Asynchronous I/O in DuckDB: Work, Thread, Work by mahogany on Sunday August 16, 2026 @08:52AM (#49319596)
Note that SET memory_limit is a soft limit and can be completely ignored for some tasks, so this wouldn’t be the same as having a limit on physical memory.
Re:Asynchronous I/O in DuckDB: Work, Thread, Work by michaelmdresser on Sunday August 16, 2026 @11:36AM (#49321025)
In past experience with DuckDB I have found this soft-limit distinction to be very important.