News for hackers. Stuff that matters.
Re:Apache Kafka Goes 1.0 by lmsp on Wednesday November 01, 2017 @08:55PM (#15606944)
another point added to 'rebalancing' -- when kafka rebalances the partitions, it has to copy all the data for the partitions that are moved around. it might not be a big problem when retention is small. however it is pretty worse when retention period is longer, rebalancing is going to exhaust all the bandwidth (both network and I/O) in the cluster. people don't realize the fact until they want to grow the cluster (adding more brokers) to support increased traffic.

There are no comments on this comment yet.