Evaluating key-value and document stores for short read data

Re:Evaluating key-value and document stores for short read d... by daeken on Monday May 18, 2009 @11:53PM (#616209)

It surprises me that Tokyo Cabinet is so slow to load, as I've had great success with it on my own, but I've not used Tyrant/pytyrant. I'd love to see some information on where the bottlenecks are in the loading process, as it seems to me that there's something funky going on that can likely be improved pretty easily.

Edit: He notes in the comments that a switch to pytc (direct Tokyo Cabinet bindings rather than going through Tyrant) would improve things, but that he wanted to keep it distributed. I'd still like to see where the bottlenecks are, though.

[ Parent | on HN ]

Re:Evaluating key-value and document stores for short read d... by greendestiny on Tuesday May 19, 2009 @12:29AM (#616251)

It was probably compressed, hence the size load/time tradeoff.

[ Parent | on HN ]

Re:Evaluating key-value and document stores for short read d... by chapmanb on Tuesday May 19, 2009 @08:12AM (#616615)

Yes, that's right. For the test I ran the tyrant server with:

ttserver test.tcb#opts=ld#bnum=1000000#lcnum=10000

d specifies a deflate encoding. The type of database may also make a difference on loading times; in this case it was a B+ tree.

[ Parent | on HN ]