News for hackers. Stuff that matters.

A comment on Dragonflydb – A modern replacement for Redis and Memcached. Parent comment

Re:Dragonflydb – A modern replacement for Redis and Memcached by romange on Monday May 30, 2022 @01:46PM (#31561542)
Yes, I can confirm that Memcached can reach similar performance as DF. However, one of the goals of DF was to combine the performance of Memcached with versatility of Redis. I implemented an engine that provides atomicity guarantees for all its operations plus transparent snapshotting under write-heavy traffic and all this without reducing the performance compared to memcached.

Having said that, DF also has a novel caching algorithm that should provide better hit rate with less memory consumption.

[ Parent | on HN ]
Re:Dragonflydb – A modern replacement for Redis and Memcached by xiphias2 on Monday May 30, 2022 @02:25PM (#31561977)
Do the benchmarks stress test those atomicity guarantees?

Get/set operations look like they don't need it.

[ Parent | on HN ]
Re:Dragonflydb – A modern replacement for Redis and Memcached by romange on Monday May 30, 2022 @02:41PM (#31562177)
You are correct - GET/SET do not require any locking as long as they do not contend and they do not in those benchmarks. You are right that for MSET/MGET you will see lower numbers. But still it will be much higher than with REDIS.

This is our initial release and we just did not have resources to showcase everything under different scenarios. Having said that, if you open an issue with a suggestion of a benchmark that you would like to see I will try to run soon...

[ Parent | on HN ]
Re:Dragonflydb – A modern replacement for Redis and Memcached by xiphias2 on Monday May 30, 2022 @05:59PM (#31564088)
No worries, I trust you to do the right thing, just measuring something that Memcached can do fast anyways can be a misleading benchmark.
[ Parent | on HN ]
Re:Dragonflydb – A modern replacement for Redis and Memcached by anitil on Tuesday May 31, 2022 @12:05AM (#31566005)
Do you know what tooling is available for testing atomicity guarantees?
[ Parent | on HN ]
Re:Dragonflydb – A modern replacement for Redis and Memcached by romange on Wednesday June 01, 2022 @03:05AM (#31578988)
The best tool out there is probably jepsen.io but it requires domain knowledge to properly operate it. The guy behind it is a beast!

Anyway, I wrote lots of unit tests to cover those atomicity issues. You can also write a custom python/nodejs/golang/... scripts that simultenusly write and read from the same multiple keys in such way that some invariant is preserved. For example, "mset x {i}, y {i}" for random `i` and in parallel do "mget x y" and to check that the response returns same values. You can also test this for other families using transactions like "MULTI; lpush x ${foo}; lpush y ${foo}; EXEC" .. and then similarly test that x and y have exactly the same lists.

[ Parent | on HN ]
Re:Dragonflydb – A modern replacement for Redis and Memcached by anitil on Thursday June 02, 2022 @08:14AM (#31593881)
Ah thanks I had heard of and then forgotten about jepsen. I had considered what you described but I thought there might be a more targeted approach where you can specify your invariants (like "x can only be in one of two states" or "pushing alway increases the length of a list and popping always reduces it if it's non-empty") and get the tool to try to break them.

I wonder if there could be a tool that instruments in the same way that afl does to try to detect races or inconsistent states.

[ Parent | on HN ]