Check out BerkeleyDB, either in its original incarnation or 'Java Edition'. On one machine, it would seem to fit your bill: uses RAM cache as much as possible, persists to disk as necessary, various tweakable options to try to either minimize IO/seeks or maximize consistency/safety.