News for hackers. Stuff that matters.
Re:The problem with conventional databases by rtm on Tuesday April 24, 2007 @04:53AM (#16264)
You can configure a drive to delay writing to the disk surface, and instead just write into its cache, until some later point when it's convenient to write the surface. But the reason a DB issues a write to the disk is that the DB needs the data to be recoverable after a crash before the DB can proceed. So DBs cannot easily use the disk's write-to-cache feature; the disk's cache is no more durable than main memory.

You might imagine that the disk would write-cache only an amount of data that it could write to the surface with the energy stored in its capacitors after it detected a power failure. But this is not the way disks work. Typical disk specs explicitly say that the contents of the write-cache may be lost if the power fails.

You may be thinking of "tagged queuing", in which the o/s can issue concurrent operations to the disk, and the disk chooses the order in which to apply them to the surface, and tells the o/s as each completes so the DB knows which transaction can now continue. That's a good idea if there are concurrent transactions and the DB is basically doing writes to random disk positions. In the log-append case we're talking about, tagged queuing is only going to make a difference if we hand lots of appends to the disk at the same time. In that specialized situation it's somewhat faster to issue a single big disk write. You need to defer log flushes in either case to get good performance.

Re:The problem with conventional databases by dfranke on Tuesday April 24, 2007 @04:58AM (#16266)
You might imagine that the disk would write-cache only an amount of data that it could write to the surface with the energy stored in its capacitors after it detected a power failure.

That's exactly what I assumed, at least for high-end disks. Any idea why they don't do that? It seems like a pretty trivial hardware feature that would save an awful lot of software complexity.