Re:Publishing with Apache Kafka at The New York Times
by elnygren on Wednesday September 06, 2017 @08:46PM
(#15188335)
if you read more closely, this article is not about the amount of data, but the way how it is managed.
[ Parent
| on HN ]
Re:Publishing with Apache Kafka at The New York Times
by tlberglund on Wednesday September 06, 2017 @10:00PM
(#15188622)
Precisely. While "all NYT data since 1851" sounds like a lot, and >8660 days sounds like a long retention period for a Kafka topic, this--like most systems in the world--is not a Big Data application. One of the key insights from the post is that there are interesting architectural considerations that have nothing to do with data size that make immutable logs a good idea.