Re:Finding Kafka’s throughput limit in Dropbox infrastructure
by Rebelgecko on Friday February 01, 2019 @12:23PM
(#19056136)
The compression article was really useful for me when I was trying to squeeze a bit more performance out of our brokers at work. Any plans to revisit the article now that Kafka has zstd support?
[ Parent
| on HN ]
Re:Finding Kafka’s throughput limit in Dropbox infrastructure
by nosequel on Friday February 01, 2019 @12:58PM
(#19056549)
They talk about zstd in the post and they were the ones who added support to Sarama.
Re:Finding Kafka’s throughput limit in Dropbox infrastructure
by halbritt on Friday February 01, 2019 @06:39PM
(#19059861)
I'm using zstd with my kafka implementation and it works great. v2.1 is required. We're using librdkafka, which works, but I believe we had to compile it in to get that and idempotent producer.