News for hackers. Stuff that matters.
Re:Finding Kafka’s throughput limit in Dropbox infrastructure by halbritt on Friday February 01, 2019 @01:21PM (#19056810)
I understand that you can increase single-broker throughput, but one wonders what use case requires that? That'd be warranted in the case of single-partition topics which are necessary to preserve strict ordering. If one has ability to create enough partitions to mitigate hot spots, I'm not sure I see the need.

My org has gone a different direction. We're using spinning rust in GCP and deploying on kubernetes to make scaling out easy. Standard disk provides surprisingly good performance (relative to AWS) and we can easily ingest 10gbps with 20-30 brokers.

Re:Finding Kafka’s throughput limit in Dropbox infrastructure by anotherhue on Friday February 01, 2019 @02:07PM (#19057365)
This was a three node cluster, with 3 replicas.
Re:Finding Kafka’s throughput limit in Dropbox infrastructure by halbritt on Friday February 01, 2019 @05:41PM (#19059475)
My question, perhaps poorly articulated was why would someone focus on improving single-node performance vs. cluster-wide performance? I wasn't implying that one would want to run a single-node cluster.