Re:Apache Kafka Goes 1.0
by manigandham on Wednesday November 01, 2017 @01:10PM
(#15603338)
Low-volume = relational database. They remain the gold standard in usability and tooling. Backups and high-availability and consistency are well solved.
Also low-volume on modern servers can easily get into 10k+ transactions per second so you have plenty of performance potential.
Don't bother with Kafka and other systems unless you really have the scale of 100k+ events per second or otherwise need to start splitting up your app into several layers. That recent NYTimes article about Kafka for storing their entire dataset of 100gb is exactly the wrong way to use it.
Also low-volume on modern servers can easily get into 10k+ transactions per second so you have plenty of performance potential.
Don't bother with Kafka and other systems unless you really have the scale of 100k+ events per second or otherwise need to start splitting up your app into several layers. That recent NYTimes article about Kafka for storing their entire dataset of 100gb is exactly the wrong way to use it.