daniele_dll

GET and SET (and similar) are easy, let me know if you keep the same performances once you start to need RMW operations or even transactions to deal with multi keys commands :)

Also going faster than Redis doing only GET and SET is fairly easy, the very very very early versions of cachegrand (pre 0.1) were doing well over 100M/s of GET and SET on a 32 core / 64 threads machine.

Once you start to chuck the rest to make it actually "usable" it's where you start to massively lose performances.

Thank you! I worked super hard to keep the token usage as low as possible. It's also possible to drive mobile browsers via the MCP, although I really tested only chrome, I have done a bunch of optimization for token usage also on that front but never pushed the testing too much further, if you want to experiment and provide feedback I will be more than happy to check things out!

Well if you check the screenshot you will notice that LinkedIn was using about 10x more memory for a few tabs ;) The most insane one is at +15GB lol

I felt "reductive" reading that LinkedIn was using <3GB for 2 tabs given my experience.

I posted because I saw that a few days ago but 2.4GB felt "reductive" considering what I experienced on my machine ;)

It's always nice to see someone discovering that mmap is not built for performance but for easyness of data-on-disk management with reasonable speed.

Meet Cachegrand, an open-source (BSD licensed) Redis-compatible key-value store designed for speed. Our latest release offers significant improvements, allowing you to use Cachegrand in environments without clustering, while still maximizing performance.

Get started quickly using our Docker image and enjoy the benefits of one of the fastest Redis-compatible caching platforms out there. Cachegrand optimizes performance for both physical and virtual hardware.

Here our github repository https://github.com/danielealbano/cachegrand

We're committed to transparency, so we've made all the raw benchmark data available at https://github.com/cachegrand/cachegrand-benchmarks. We'd appreciate your feedback! Give Cachegrand a try and experience the difference today.

This is a valid reason, but makes sense for code bases where the generation takes tens of minutes or more not seconds.

Importing a C file into a Python session is a practice that can ---easily--- cause ---a lot--- of headaches, the packages required to build the code as python library and make it importable will vary between systems and make it hard to debug in case of crashes. I personally would distribuite the pre-compiled binaries, CI + docker makes that much easier.

The fact that "can be done" doesn't necessarely mean that "it's the right way" to do it.

Going back to embedding auto-generated code, focusing on this specific case, are you telling me that a line in the requirements.txt, on a machine where the user is expected to have access to a compiler, is the reason to keep auto generated code in the repository? Sorry, personally it seems a flaky reason.

Also, the very fact that you mention that users don't have superuser privileges should be even more of a reason to use virtual environments, where you don't need super privileges, or if you can't or want to use virtual environments you can still have multiple paths in the PYTHONPATH env variable and you can install packages in a specific paths leveraging the --target of pip or the PYTHONUSERBASE env variable, solving the issue as well.

I would prefer to suggest my users to improve their runtime environment than enable them to improve the chaos.

Fully agree with the principle but maybe let's not call Redis "well-written" :)

Redis is a great piece of software because of how much it impacted the "evolution" of Internet not because of the code quality, Redis's code resemble spaghetti code, a lot.

I will push myself and say that nowadays people would get fired for writing code like that!

And before someone start to throw stones (just for the sake of it), here a few examples: - Redis source code is FULL of GIANT files with thousands and thousands of lines of code, decoupling and separation of concerns are basic things! A few examples -- https://github.com/redis/redis/blob/unstable/src/module.c -- https://github.com/redis/redis/blob/unstable/src/cluster.c -- https://github.com/redis/redis/blob/unstable/src/redis-cli.c -- https://github.com/redis/redis/blob/unstable/src/networking....

- talking about auto generated, the repo contains commited auto generated code, a valid reason though really slips my mind (yes, of course I can think of a few, but ... really? :)))

- https://github.com/redis/redis/blob/9c7c6924a019b902996fc4b6... ... that struct feels like an "Italian minestrone", basically a kind of soup where you throw in, literally, all the vegetables you have around and it's great for a soup, a bit less for software engineering though :)

- the naming structure is a bit random at times, there is a mix of camel case, camel case mixed with underscores from time to time, pascal case

This kind of staff might be alright for a 0.3 or 0.4, maybe even for a 1.0, definitely not for a 7.x that has had 15 years to evolve :/ Even in cachegrand, that hasn't got to a v0.2 yet, I tried to avoid these kinda of dramas, in C is quick and easy to get there if you are not careful.

There are plenty of other small things but most of the big dramas should be there.

Curious to discover how the hastable implemented in cachegrand ( https://github.com/danielealbano/cachegrand ) - my opensource Redis compatible Key-Value store - can deliver up to 112 million GET op/s and up to 85 million UPSERT op/s?

I have just pushed the new documentation! https://github.com/danielealbano/cachegrand/blob/main/docs/a...

When tested with memtier_benchmark, using the Redis protocol, cachegrand itself, on the benchmarking hardware, thanks to the implemented hashtable can reach up to 5 million GET op/s and up to 4.5 million UPSERT op/s without batching, with it up to 60 million GET op/s and up to 26 million UPSERT op/s!

I am empathetic as well but...

Let me start saying that no one prevents a company from selling the software with licenses allowing to access the source code, companies even provide the code under the GPL because the BSD or the GPL nowhere state that the code has to be provided for free, big companies have provided the code under open source licenses behind fees for quite a while so far (redhat is one of the examples).

The BSL certainly allows you to access the code without having a license but I would dare to say that most of the companies out there, especially for critical components like this, prefer managed and supported solutions, for which rights to access the source code can be granted.

But in general it's absolutely true that it allows accessing the source code without having to buy the license and this is certainly a plus for the final user.

But that's it :)

I will not say that I know the legal language but I will say that the legal language really differs form what we perceive and a judge or a court perceive and the interpretation is dictated by the national law.

The sentence

> Effective on the Change Date, or the fifth anniversary of the first publicly available distribution of a specific version of the Licensed Work under this License, whichever comes first, the Licensor hereby grants you rights under the terms of the Change License, and the rights granted in the paragraph above terminate.

Use the word "version" which is broad and generic, can be read as "the commit makes a new version" or "the intent for the developer to release a new version".

Do, again, no certainty really ok the long term.

Going open source require sacrifices, I have given for free more than 1 year of research and development I did to build the technology behind the hashtable implemented in cachegrand and I did because I believe that true technological advancement should be available to anyone and meanwhile my work is not a game changer it my small bit to try to help.

That's why when I see a license like BSL pointed out as open source or "quasi" open source I do feel the need to give some context.

Try to imagine if Linus at the time would have decided to give Linux under BSL... Nowadays would be very different!

It's definitely not entirely true, the license grants that "the current version" under "the current license" will be under the Apache license in 5 years that translates to in 5 years dragonflydb 0.6.0 / 0.7.0 will be under the apache license. That's the only guarantee.

The change date can be changed at any time, the license allows it, almost all the projects I saw using the BSL actually use a sentence like "X years from the release", on top of this it can be even removed and if that would be the case than the paragraph below would apply, which clearly refers to 5 years from the released versions.

> Effective on the Change Date, or the fifth anniversary of the first publicly available distribution of a specific version of the Licensed Work under this License, whichever comes first, the Licensor hereby grants you rights under the terms of the Change License, and the rights granted in the paragraph above terminate.

5 years old software is a dead software :)

The only reason for adopting the BSL is to prevent companies like Amazon to take ownership and don't give back to the project (e.g. elastic), which is totally understandable.

Do you think that future releases, much more feature-complete and flexible, will be given under the apache license as they are meanwhile less feature complete versions aren't? Looking at the facts, I would say the answer is very likely no.

Would you build something in 5 years based on dragonflydb 0.7.0 when most likely there will be a v2.0 or even a v3.0 out there?

When it comes to non open source licenses and companies it should be clear that the companies want (and need) to take care of their own interests: the BSL "change date" is a way to be able to say "yes but it will become OSS" so you can keep using and contributing, like the sop you give to a kid to stop him from crying.

Thanks!

I also promise to document more the actual inner workings of the various components, I have some scattered documentation here and there which will help me in the task!

I replied in a few posts in the thread with some details, if you want more there is also this presentation I did at the London performance summit https://www.youtube.com/watch?v=ZSpxuWQK7sM

Most of the performance exploits for the hashtable (which is the driving engine here) are explained after minute 8.30 (although I would suggest you to watch the entire presentstion).

I will present my work at the p99conf as well.

Please don't confuse free source with open source, there is quite a massive difference.

There is a reason for which the BSL is not a recognized open source license.

It is early stage in the sense it only support commands that operate on the strings and some on the key space.

Other commands are planned to be added soon but the ones already implemented cover already a lot of different use cases.

Let me answer and then give you some background for my question.

> Which says you estimate 50-100 machines to saturate one "low-core count VM" Redis. But now you say you meant 50-100 machines for Redis servers?

No, I was referring to machines running business logic functionalities using Redis as part of their processing pipeline.

Said that, the reason for which I was asking for some numbers was to figure out if my expectation of "very often you need more room for the data than better performance" was making sense.

Because cachegrand will be able to store data on disk and to handle stream writes as well allowing to leverage the time series db, once fully implemented, it will able to cover a number of different needs. This combined with the ability to run Webassembly means that you will also be able to run whatever you want directly in place and data will not have to go out and in from multiple systems to be processing making the pipelines much faster and cheaper to run.

Of course it doesn't take 1 day, especially because I am working on cachegrand during my free time (e.g. At night or over the weekends).

In case you are interested, here there is a presentation I did for London Performance Summit https://www.youtube.com/watch?v=ZSpxuWQK7sM

For the hashtable part you can jump to minute 8.40.

This year I will present my efforts at the P99Conf, the content will be similar.

Yes, I was planning to bring it in after a few releases and focus on more commands right now but I was asked to make it more production ready so the next release I will bring in: - Active-Active replication - memory control (with key eviction using an LRU) - RDB load & save (or at least RDB load) - the ability to run cachegrand as slave of a simple redis cluster (e.g. sharded but active-passive replication), so will be possible to introduce cachegrand slowly and test it out without impacting a whole deployment.

Absolutely, I was planning to introduce the Active-Active replication in a few releases but I changed the plan to do it for the v0.2, together with a memory control and an LRU to evict keys if it's necessary to free up memory, with RDB load support and the ability to work as slave in a Redis cluster, so it will be possible to join it to a cluster and having running to be used for other purposes if wanted (e.g. analytics or similar).

In general is pretty stable, I let it run for hours under memtier_benchmark hammering it, although I relocated in March and all my stuff is still packed in a warehouse (it's impossible to find an apartment in Zurich, Switzerland) but I am getting a new 2 x AMD EPYC 7551 with 512GB of ram to have plenty of room for testing and benchmarking with local hardware instead of asking favours :)

A bit from everything: the hashtable leverages the CPU caches and the cachelines, it also uses SIMD instructions to search very quickly the hash, once a bucket in the hashtable is identified, only the relevant bits in the hash are compared, on top of this there is the custom memory allocator, fibers and io_uring is of great help network side, etc.

The multithreading helps to scale the access to the hashtable vertically almost linearly.

The use of the fibers make everything easier because cachegrand doesn't have multiple threads running on the same core fighting for the same resources.

On top of this (and thanks to not having multiple threads fighting for the same resources on the same core), the hashtable uses user-space spinlocks (that are "fake" spinlocks as they can't prevent the kernel to preemept, although there are ways to do it) and the "contention" is spread all across the hashtable. The spinlocks though are only used to change the hashtable (with UPSERT, DEL and Read-Modify-Write operations), they are not required to read from it.

Let's say that thread A, B and C, want to access key1, key2 and key3.

If you an hashtable of 10000 buckets, cachegrand will basically split it in chunks of 14 elements and each chunk will have its own localized lock, so there will be about 714 different possible locks. In general, unless you are writing always to the same key, the writes will be distribuited across all the hashtable hitting different locks so it will not be necessary to wait.

Of course I considered different approches a few years ago when I was researching the hashtable including sharding it across multiple threads and let a thread own a slice of it but decided not to go down this way because instead of having 714 locks, like the case above, you will need to use queues to pass messages between the threads (a bit like Scala does) and "focus" the contention on limited number of memory areas (e.g. if you have 4 threads you will have 4 queues and focus the contention on these 4 queues which is bad). Of course you can use atomic operations to push and pop elements from these queues but it will not be able to scale up as the contention will make this system tremble under heavy load.

cachegrand, thanks to the localized spinlocks, can hit 60 million GET and 27 million SET per second on hardware used for benchmarking.

Currently both, the TSDB is a PoC right now.

The timeseries will depend on the keys, as it's the historical sequence of values, so it will be replicated and merged on the nodes that own that specific key in active-active replication with the required replication mode.

Although not mandatory, for the active-active replication mode cachegrand will provide a front-end proxy that "understands" which is the correct node for a key and send the necessary data to always the same node (or subset of nodes, depending on the configuration).

The replication itself will be last-write-wins, if the order of the writes matter it will be important to write always to the same node (very common pattern to reduce syncing and locking on the replication side).

That comment doesn't really apply to cachegrand and the benchmarks shared on the project page, cachegrand can run in single thread mode and there is a benchmark for that as well (the first column of the chart).

But honestly, performances aren't the "only" goal and they aren't the main goal neither although to get where we want to be of course we need extremely good numbers! :)

If I might ask, as I guess from your comments you are using Redis or a compatible platform, which are your numbers? Specifically I am referring to number of servers / vms for Redis, total core count, total memory available, total memory usage.

Thanks!

Very good point but currently cachegrand doesn't have an interface for memcache, although probably would take about 1 week to implement it.

In general I am pretty sure cachegrand would be faster than memcache easily but I will add it to the todo list as it makes sense.

Thanks for pointing it out!

I would love to sit and write more documentation but I am doing this work during the night and/or over the weekends so sorry for not having it but I promise I will slowly slowly start to put together something more than a "TODO", even if it's a general intro.

The timeseriesdb is "half" a consequence of having a Write-Ahead-Log that is split in chunks and it's chained. I am saying "half" because the other one is the future addition of secondary index to make it possible and easier to query the internal db properly.

I know that we are talking mainly about Redis right now, but that's just the tip of the iceberg and my long term vision is to build a much more complete and flexible platform which can easily handle streams (e.g. via a Kafka interface) and/or allow you to run more evolved data processing via WASM (e.g. I want yo make possible to calculate rolling window averages in a time-sensible fashion :)).