The trouble is that naive mmap doesn't give you the control over write ordering that you need to implement reliability.

Sadly, about 50% of database design is trying to ensure the thing has a decent chance of starting up again if it crashes or loses power. This mostly consists of fighting file systems and disk caches.

I used to work on a system that used mmap for basically everything, including a proprietary database that processed financial transactions. The original production system ran on a commercial Unix. (It's been about 15+ years, but I think it was AIX.)

At one point, we ported it to a different Unix platform which had slightly different mmap semantics. It looked like everything worked, except for one minor detail: the data was never actually synced to disk. Ever. Until shutdown. Unfortunately, since the system ran 24/7, it effectively never synced. First time the system had a power failure, there was massive data loss on startup.

Oops.

(We were able to recover through log replay...)

> (We were able to recover through log replay...)

I call that a win! What was the logging mechanism--something bespoke? What was your solution?

It was bespoke: a proprietary system that essentially used an append-only file containing each transaction request and response.

Very nice. Good planning.

That sounds like a terrible idea even if it were 100% reliable. When you shut down it will have to flush the entire memory at once. If you're syncing to a hdd it can mean several minutes of downtime.

A correctly working system will "flush behind", so there should be no difference between a clean shutdown and a power loss. If there's several minutes of unsynced data that's data that will be lost on power off.

Just going to chip in here to mention LMDB, which essentially provides an mmap btree database with single writer + multi-reader snapshot concurrency, ACID transactions, and is extremely robust.

If you think something may benefit from a shared memory data store, lmdb may be worth considering as a fast, reliable, high concurrency alternative to reinventing wheels with raw mmap + manual sync.

Not the OP here, but why the downvotes? Is this person off-base, or is that a poor solution?

The poster's comment reads a bit like an advertisement, but LMDB would be a good solution for persisting data, in my opinion. It has a pretty simple C API and it's been described by its authors as "crash-proof".

There's `msync`

This is why msync is for. You can sync page by page and ensure ordering yourself.

You think you can, but you can't! Linux always sync's the whole file; see https://lwn.net/Articles/502612/ for a discussion of why fixing this horrible bug is too dangerous.

"You think you can, but you can't!"

Welcome to systems programming....

This is not true after Linux 3.16, commit 7fc34a62ca44 ("mm/msync.c: sync only the requested range in msync()").

Why not really? From the sqlite docs:

  Beginning with version 3.7.17 (2013-05-20), SQLite has the option of accessing disk content directly using memory-mapped I/O and the new xFetch() and xUnfetch() methods on sqlite3_io_methods.

I would say its less that you can’t or shouldn’t use mmap, and more that its not as simple as throwing mmap at it - if you care about crash recovery/not corrupting data (which most databases should care about), but that certainly doesn’t mean that mmap can’t be a part of the solution.