The write amplification in LMDB is bounded by the height of the tree, so it's O(logN) with N=number of records. The write amplification in all WAL-based designs is O(N) where N=size of the records. With a 4K page size, to get a write amp of 64K in LMDB would require a tree depth of 16. At 100 bytes per record, 40 records per page, that would require 40^16 records, or 42949672960000000000000000 records. IMO your figure of 64K is vastly overestimated.
I've done more detailed investigation of write amplification in various DB engines here http://symas.com/mdb/ondisk/ and you can see that the break-even point relative to WAL-based engines is around record size of 2KB. Above that size LMDB has lower write-amp than WAL-based approaches. Coincidentally (hah) LDAP entries tend to be >= 2KB in size.
But most importantly for real-time services - LMDB write latency is tightly bounded, and never suffers from stop-the-world compaction pauses. This was a design requirement from day 1. (See section 3.2 of the 2011 LMDB design paper http://symas.com/mdb/20111010LDAPCon%20MDB.pdf ) E.g., see the comparison of write latency in LMDB vs HyperLevelDB here http://symas.com/mdb/hyperdex/#100M - we can reliably compute/predict LMDB's I/O latency and our prediction perfectly matches the real world results - 33ms avg latency on a disk with 16ms avg seek time. You cannot do that with any of the other DB engines.
You were right up till that - it reuses pages as soon as they are safe to reuse, not waiting to get to the end of the file.
Anyway, yes, write amplification is a significant factor, in any DB design.
100M records, 1M ops
MinLatency(us) AvgLatency(us) 95th%ile(ms) 99th%ile(ms) MaxLatency(ms) Runtime(sec) Throughput(ops/sec) CPUtime (mm:ss)
LMDB update 256 33565 130 209 637 8385 119 4:21.41
LMDB read 215 33493 130 207 817
LevelDB update 241 63660 231 385 20370 15863 63 17:27
LevelDB read 188 63250 230 383 7904
(It's formatted properly if you follow the link)Also look at the latency distribution compared to MySQL's InnoDB http://symas.com/mdb/memcache/
From watching Rich's talks on Datomic, the argument seems to be that for many problem domains, we can spend the cost of immutability to get a much nicer development environment.
If you're already sharding your DBs in a frantic effort to keep your web startup scaling, obviously switching to an immutable database is a bad idea.
But if you can fit every transaction your company has ever done in memory, why wouldn't you go append only?
While the latter might be a niche market today, if Moore's Law keeps up for another few decades it'll be the common case. I'm not clicking on more links than I was 15 years ago, but I've got three orders of magnitude more memory.
The original post isn't exactly wrong, it's just something of a straw man. We live in an era of terabyte disks. Cold data storage is cheap.