Re:RethinkDB (YC S09): MySQL Storage Engine Built From The G...
by cperciva on Tuesday July 28, 2009 @10:04PM
(#729300)
Congratulations -- but keep in mind that cleaning (aka. garbage collection) is the hardest part of any log-structured storage system. I learned this lesson first-hand when writing tarsnap. :-)
[ Parent
| on HN ]
Re:RethinkDB (YC S09): MySQL Storage Engine Built From The G...
by leif on Tuesday July 28, 2009 @10:13PM
(#729309)
Thanks; we're doing GC over the next few days (starting...now actually).
Re:RethinkDB (YC S09): MySQL Storage Engine Built From The G...
by cperciva on Tuesday July 28, 2009 @10:24PM
(#729320)
Full GC (removing everything which is not needed to read the current DB state), or partial GC (removing old metadata such as indexes, but leaving behind data which has been deleted/modified)?
The former is much harder -- it's a significantly complex task just to figure out which data records are still live.
Re:RethinkDB (YC S09): MySQL Storage Engine Built From The G...
by coffeemug on Tuesday July 28, 2009 @10:47PM
(#729343)
Full GC. In the framework of what we're doing, it's not very complicated (we think). This is because of our indexing scheme (we can simply walk the index tree while the database answers queries). We hope to release more info on this shortly.
Re:RethinkDB (YC S09): MySQL Storage Engine Built From The G...
by cperciva on Tuesday July 28, 2009 @11:30PM
(#729384)
Cool -- good to see that you're thinking about this stuff ahead of time rather than sitting down to write code and suddenly realizing that you painted yourself into a corner. :-)