Re:HyperDex: A searchable distributed key-value store
by DennisP on Monday June 25, 2012 @10:32AM
(#4156661)
I'd be very interested in your threaded-discussion sample. Couldn't find it on your site, do you have it online somewhere?
Re:HyperDex: A searchable distributed key-value store
by egs on Monday June 25, 2012 @10:38AM
(#4156690)
Here it is http://gibbr.org/. An undergrad implemented this on HyperDex in about half a semester.
Re:HyperDex: A searchable distributed key-value store
by rescrv on Monday June 25, 2012 @10:38AM
(#4156691)
We have a demo of an earlier version deployed on http://gibbr.org/. We currently are not releasing the source, but we may make an example application using the same design we use in the real app.
Redis offers many datastructures as well, but it has a limited architecture. If you want to run multiple Redis instances, you must run them in a master-slave configuration with no guarantees when the master fails. HyperDex can withstand such faults while guaranteeing linearizable semantics. Further, we've got some changes in the pipeline that will enable us to support every datastructure operation Redis supports (BRPOPLPUSH, I'm looking at you) with horizontal partitioning across multiple machines.
Hyperspace hashing stores multiple copies, but this is configured by the user. In our applications we've had at most two copies.
You can find more about looking up non-primary data attributes in the tutorial (http://hyperdex.org/doc/tutorial/#creating-a-new-space). It enables you to search over attributes of an object without having to maintain secondary indices.
We've built a sample threaded-discussion application on top of HyperDex. We're also in talks with some astronomers who would like to use it to analyze data from a radio telescope. The geometric nature of hyperspace hashing makes it easy to perform queries such as, "retrieve all objects in this cone through space." For any application where you perform secondary attribute search, HyperDex is the way to go. For other applications where you want high throughput and low latency with strong consistency, HyperDex is a good bet.