Dragonflydb – A modern replacement for Redis and Memcached

Re:Dragonflydb – A modern replacement for Redis and Memcached by baobob on Monday May 30, 2022 @03:28PM (#31562696)

Benchmarks done on only one metric are often misleading (and in commercial circumstances usually intentionally!). Would love to see visually what trade offs Dragonfly is making to achieve the numbers from the chart at the top of the README. If excellent technical work means there really is no trade off, that would also be a great reason to chart it visually.

Also as a Redis replacement, it's not clear what durability is offered, and for most Redis use cases this is close to the first question

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by romange on Monday May 30, 2022 @04:31PM (#31563337)

How you would suggest to demonstrate visually that there are no trade-offs?

The tradeoff the way I see it - one needs to implement 200 Redis commands from scratch. Besides, I think DF has a marginally higher 50th percentile latency. Say, if Redis has 0.3ms for 50th percentile, DF can have 0.4ms because it uses message passing for inter-thread communication. 99th percentiles are better in DF for the same throughput because DF uses more cpu power which reduces variance under load.

Re-durability - what durability is offerred by Redis? AOF ? We will provide similar durability guarantees with better performance than AOF. We already provide snapshotting that can be 30-50 faster than of Redis.

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by baobob on Monday May 30, 2022 @07:03PM (#31564561)

There is usually a tradeoff between latency and throughput, although I'm not so sure this would be true for your innovation, since you've eliminated a whole chunk of fixed overhead from the system (syscall batching). However, batching in general often implies added latency

If I recall ScyllaDB has some excellent examples of demonstrating this particular tradeoff visually. A simple option would be a scatter plot where X = latency, Y = load or similar, with points coloured according to the system under test. Probably there is a better option, but this would likely be enough to sell me at least

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by romange on Monday May 30, 2022 @07:13PM (#31564614)

Well in that case, I hope to have an good answer for you. Right below the benchmark graph I deliberately put a table that shows 99th percentile lantencies at *peak* throughput as reported by memtier_benchmark. 99th latency percentile of Dragonfly at its peak throughput. I put it here as well

  op r6g c6gn c7g
  set 0.8ms 1ms 1ms
  get 0.9ms 0.9ms 0.8ms
  setex 0.9ms 1.1ms 1.3ms

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by lclarkmichalek on Tuesday May 31, 2022 @05:33AM (#31567551)

There are also things that are unmitigated improvements.

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by pcthrowaway on Monday May 30, 2022 @05:55PM (#31564049)

Does Dragonfly have a timeseries extension? Or does it support extensions?

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by romange on Monday May 30, 2022 @07:16PM (#31564631)

It does not. If you refer to Redis Modules we are not planning to support those - we can just implement same features within DF.

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by thayne on Monday May 30, 2022 @04:17PM (#31563231)

Presumably, most of the performance benefit comes from using linux io_uring, which limits it to using recent linux kernels.

Of course, it is also possible there are situations where it doesn't perform as well.

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by YarickR2 on Monday May 30, 2022 @03:39PM (#31562812)

Tradeoff is inability to run on anything besides recent Linux kernels

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by miohtama on Monday May 30, 2022 @05:44PM (#31563963)

Does anyone run production Redis/memcached outside Linux any case?

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by citrin_ru on Tuesday May 31, 2022 @05:33AM (#31567554)

Redis/memcached are used in production on FreeBSD servers (and other BSD systems).

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by genewitch on Monday May 30, 2022 @08:50PM (#31565116)

Probably not many in production, but the "recent kernel" is going to be the more important part.

I'm not hip to how much new stuff is backport-able, so this may preclude Ubuntu 20.04, for instance. You lose the "LTS" part if you compile your own kernel, if you manage to make it functional at all.

Note: I never use kernel modules due to the issues rhel/debian and I have had with such things in the distant past.

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by arinlen on Tuesday May 31, 2022 @01:50AM (#31566484)

> I'm not hip to how much new stuff is backport-able, so this may preclude Ubuntu 20.04, for instance. You lose the "LTS" part if you compile your own kernel, if you manage to make it functional at all.

I'm not sure that concern is justified. It seems io_uring was pushed as part of the 5.1 linux kernel release, and Ubuntu 20.04 LTS seems to have been shipped with 5.4.

https://packages.ubuntu.com/search?keywords=linux-image-gene...

Also, a quick Google search pointed to io_uring patches for Ubuntu 18.04 LTS.

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by 10000truths on Tuesday May 31, 2022 @12:46AM (#31566196)

The kernel has its own LTS releases, and running Linux in the cloud is a well tested use case. Any library not available in your distro of choice can be built from source and tracked manually, which usually works well enough.

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by tyingq on Monday May 30, 2022 @04:14PM (#31563198)

I suppose because of io_uring?

It does sound like extendible hashing might have downsides in some scenarios also.

[ Parent | on HN ]

Re:Dragonflydb – A modern replacement for Redis and Memcached by Symmetry on Tuesday May 31, 2022 @10:39AM (#31570018)

Sad that you can't run dragonflydb on DragonFlyBSD.

[ Parent | on HN ]