Redpanda – A Kafka-compatible streaming platform for mission-critical workloads

Re:Redpanda – A Kafka-compatible streaming platform for miss... by runT1ME on Monday November 16, 2020 @12:30PM (#25114415)

I'm a fan of scylla too, but if I could go back in time I'd have recommended waiting until mid 2019 to migrate. 'Fully optimized C++ rewrite's tend to take years to become battle tested.

[ Parent | on HN ]

Re:Redpanda – A Kafka-compatible streaming platform for miss... by agallego on Monday November 16, 2020 @01:01PM (#25114867)

indeed. that's why we continuously _empirically_ prove that we are in fact safe - https://vectorized.io/validating-consistency/

there is no substitute to testing tho

there are 2 levels here. 1) raft has a proof (and a great phd dissertation from diego), but what matters is if we actually implemented it correctly. so 2) is we need to continuously test it. Denis did a lot of similar work at CosmosDB (microsoft) and has spent his career working on consensus.

Hopefully these eases some concerns.

[ Parent | on HN ]

Re:Redpanda – A Kafka-compatible streaming platform for miss... by bashcoder on Monday November 16, 2020 @12:48PM (#25114673)

In fact, we couldn't switch to Scylla prior to 2019 because their support of incremental counters wasn't yet complete.

[ Parent | on HN ]

Re:Redpanda – A Kafka-compatible streaming platform for miss... by agallego on Monday November 16, 2020 @01:02PM (#25114886)

totally. today if folks. need txns, they wouldn't be a good fit for us. What we found is about 90% of use cases are covered by the base api. For reference w/ all of the versioning there is something like 144 api calls you can make to kafka, most ppl use a small subset of those via high level clients. (java, python, librdkafka, etc)

[ Parent | on HN ]