Re:Module Hub – The Redis Modules Marketplace
by out_of_protocol on Sunday May 15, 2016 @07:29AM
(#11700503)
Weird times coming. Either way, i like the idea
[ Parent
| on HN ]
Re:Module Hub – The Redis Modules Marketplace
by jasonjei on Sunday May 15, 2016 @10:13AM
(#11700963)
I agree. I'm particularly interested in the full text search module. A lot of libraries that manipulate data structures in Redis could benefit from being able to modify data as a Redis module.
Re:Module Hub – The Redis Modules Marketplace
by dvirsky on Sunday May 15, 2016 @10:21AM
(#11700988)
Author of that module here. I'm recovering from jet lag after flying back from redisconf, and hope to open it in a couple of days. Mostly cleanup and some documentation left.
Re:Module Hub – The Redis Modules Marketplace
by itamarhaber on Sunday May 15, 2016 @10:30AM
(#11701025)
FoA here - it is gonna be awesome #FTFTW :)
Re:Module Hub – The Redis Modules Marketplace
by dominotw on Sunday May 15, 2016 @11:26AM
(#11701244)
>Inverted indexes are stored on top of Redis strings using the modules DMA method.
What does 'DMA method' mean here?
Re:Module Hub – The Redis Modules Marketplace
by dvirsky on Sunday May 15, 2016 @12:00PM
(#11701378)
I'll expand on it on the repo, but the idea is this: In redis modules, you can take a redis string key, allocate as much space as needed, and get Direct Memory Access to that key's data, with zero copy overhead. You can then resize that string and grow it.
So basically you can model data structures on top of this, provided that there is an easy way to convert offsets to pointers, or that your structs are scalar only.
In the case of the inverted index, it uses delta and varint encoding of the index records, so no real mapping to structs was needed, I just traverse the data and deserialize each entry on the fly in a pretty fast manner.
Another example was the use of this for bloom filters, that directly manipulate bits on the key.