News for hackers. Stuff that matters.
Re:Distributed, high-performance blob storage on SQLite by mightyham on Thursday November 19, 2026 @07:54AM (#50559490)
At the end of the day durable transaction commit in any OLTP database will always be slow because fsync is an expensive operation (~1000 IOPS on a decent SSD), so as you mentioned batching is the only way to make significant increases to throughput. That said, based on assorted benchmarks I've seen online, sqlite doesn't even get close to max fsync IOPS because it has a ton of overhead. Other embedded systems like rocksdb or lmdb are going to give better performance.
Re:Distributed, high-performance blob storage on SQLite by luke5441 on Thursday November 19, 2026 @09:13AM (#50560379)
Or you can do like GP with that possum and not fsync at all. It even sets synchronous=OFF, which will corrupt SQLite databases. Then it is fast because it only writes to memory. YOLO!
Re:Distributed, high-performance blob storage on SQLite by stingraycharles on Thursday November 19, 2026 @09:25AM (#50560537)
The usual answer here is a WAL + batching multiple transactions.