News for hackers. Stuff that matters.

A comment on Writing a SQLite clone from scratch in C. Parent comment

Re:Writing a SQLite clone from scratch in C by icedchai on Monday September 04, 2017 @12:01PM (#15169054)
I used to work on a system that used mmap for basically everything, including a proprietary database that processed financial transactions. The original production system ran on a commercial Unix. (It's been about 15+ years, but I think it was AIX.)

At one point, we ported it to a different Unix platform which had slightly different mmap semantics. It looked like everything worked, except for one minor detail: the data was never actually synced to disk. Ever. Until shutdown. Unfortunately, since the system ran 24/7, it effectively never synced. First time the system had a power failure, there was massive data loss on startup.

Oops.

(We were able to recover through log replay...)

[ Parent | on HN ]
Re:Writing a SQLite clone from scratch in C by tomcam on Monday September 04, 2017 @12:59PM (#15169429)
> (We were able to recover through log replay...)

I call that a win! What was the logging mechanism--something bespoke? What was your solution?

[ Parent | on HN ]
Re:Writing a SQLite clone from scratch in C by icedchai on Monday September 04, 2017 @02:28PM (#15169974)
It was bespoke: a proprietary system that essentially used an append-only file containing each transaction request and response.
[ Parent | on HN ]
Re:Writing a SQLite clone from scratch in C by tomcam on Monday September 04, 2017 @02:49PM (#15170126)
Very nice. Good planning.
[ Parent | on HN ]
Re:Writing a SQLite clone from scratch in C by imtringued on Monday September 04, 2017 @04:11PM (#15170565)
That sounds like a terrible idea even if it were 100% reliable. When you shut down it will have to flush the entire memory at once. If you're syncing to a hdd it can mean several minutes of downtime.
[ Parent | on HN ]
Re:Writing a SQLite clone from scratch in C by pjc50 on Monday September 04, 2017 @04:35PM (#15170700)
A correctly working system will "flush behind", so there should be no difference between a clean shutdown and a power loss. If there's several minutes of unsynced data that's data that will be lost on power off.
[ Parent | on HN ]