Re:Architecture of a Database System
by jandrewrogers on Friday January 23, 2015 @06:45PM
(#8938103)
To be clear, I am assuming the same hardware budget for both the on-disk and in-memory case. Ceteris paribus and all that. An in-memory model has no inherent advantages in that case because the same memory is potentially available to both for a given workload. Another way of stating it is that SSD won't slow you down.
Ironically, in practice relative performance between these two models is all over the map. Quality of implementation has much bigger impact than the abstract model. I've seen high-quality disk-backed database engines wipe the floor with in-memory implementations on the same hardware and vice versa.
Yeah, performance frequently has more to do with the workload than it does the underlying hardware.
One more thing to consider in your test case - the equivalent cost hardware for spinning rust would get you almost exponentially more storage. Server grade SSDs are still very expensive per GB.
Re:Architecture of a Database System
by justincormack on Saturday January 24, 2015 @06:02AM
(#8939340)
You can add third tier spinning rust for very little extra money, but latency may be an issue.
If, and only if, you are returning all/most of the data fetched from disk, and not just a fractional portion of the data (which is the most common use case for DBs, especially when you use stored procedures).
In the worst case, you have to scan the entire contents of a several terabyte table from disk just to get metadata about that table. The worst case is usually refactored away, but there are always cases where it is simply not possible.
I've seen some pretty silly speedups with flash (such as attaching the flash memory to DIMM slots with custom kernel drivers), but data served out of memory was still faster (if only because it doesn't require a context switch to read).