Vishnevskiy
We are already multiple times larger than Slack :)
https://blog.discordapp.com/the-discord-store-beta-9a35596fd...
Aside from the money we already make from https://discordapp.com/nitro :)
And thank you for helping make Rust awesome.
Our message storage clusters have a very large set of data that keeps increasing and using Scylla without incremental repair will suck so we are waiting on that.
And thank you again with helping us with that bug :) we really appreciate it.
We love Cassandra, and hate it at the same time.
Check out this nasty bug we got. https://issues.apache.org/jira/browse/CASSANDRA-13004
Voice issues should not be happening. Please contact our support with more information and we will gladly investigate.
Normally though since the data is replicated on 3 nodes, you can technically loose a node completely and rebuild it from the other nodes.
Please email me [email protected] I would love more details.
You are basically advocating for plugging 2 systems together, which out of the box don't provide elasticity. Or we could just use Cassandra. It is a simpler solution and Cassandra is not new tech. Aside from caching empty bucket information we have nothing sitting in front of Cassandra. It works great and the effort was minimal.
The Redis comment by jhgg was referring to our service which tracks read states for each user. We might write about that later, but it's not as interesting. The most interesting about that experience was reusing memory in Go to avoid thrashing the GC.
We care about seamless elasticity for our services which Redis doesn't provide out of the box except with Redis Cluster which does not seem to be wildly adopted and forces routing to clients.
I guess everyone makes their own tradeoffs though. This has been working wonderfully for us.
Riak is not a good model since its more a blob store and we wanted to simply range scan through messages rather than sharding blobs (Cassandra is REALLY good at this).
HBase would have been fine for this model, but the open source version of HBase has much lower adoption than Cassandra so that was a big factor. We also don't care about consistency and HBase is a CP database, we prefer AP for this use case. As far as using GCP's BigTable (HBase compat), we made this decision before we moved to GCP, but we are also not fans of using platform lock-in. While BigTable has the same API as HBase we would hate to go to an less widely adopted version where we have a hard time getting community support if we decided to leave GCP.
Hope that helps.
Add me on Discord if you ever wanna chat, Stanislav#7943
Add me on Discord if you wanna chat Stanislav#7943
Also ScyllaDB is a new product and it would be crazy to start off with it. We plan to run a long-term double write experiment before we are comfortable with using it as a primary data store.
You can also search past messages and the blog post ends with saying the follow-up will be about search. Search is not rolled out to everyone but will next week.
https://support.discordapp.com/hc/en-us/articles/11500046858...