Re:Don't use Hadoop when your data isn't that big (2013)
by e12e on Wednesday May 24, 2017 @08:27AM
(#14409482)
> Not sure of exact numbers, but I think we're doing 4-5m writes per second across one Kafka cluster, and around 1m writes per second against another.
I'm not saying that's the case with your system, but my immediate thought when I see those numbers is: how much of the value from those 4m/s writes could you get with a system that did something like 100 w/s? Either through sampling/statistical methods, reduce "write amplification" or simply looking hard at what is written and how often :-)
I'm not saying that's the case with your system, but my immediate thought when I see those numbers is: how much of the value from those 4m/s writes could you get with a system that did something like 100 w/s? Either through sampling/statistical methods, reduce "write amplification" or simply looking hard at what is written and how often :-)