News for hackers. Stuff that matters.
Re:Don't use Hadoop when your data isn't that big (2013) by salex89 on Tuesday May 23, 2017 @12:20PM (#14402524)
As you said, usually it is enough, but people tend to (over)design and plan for scale, and Kafka does become a better solution above a limit. Shall you touch it, that's another topic. I wouldn't say that it is a rare occasion that you absolutely need Kafka, but a less frequent one, surely. Also I don't think it is a terrible miss to use it if you need a queue with reasonably high demands from it. You'll mostly waste operations efforts and development time. The end result will not differ too much.

Nevertheless, the case from above is a very extreme case of over engineering and premature optimization. To the limit where would qualify it as reckless.

Re:Don't use Hadoop when your data isn't that big (2013) by andy_ppp on Tuesday May 23, 2017 @12:48PM (#14402800)
Why does everyone seem to think Kafka is a queue replacement; it's a high performance immutable log with a guaranteed CAP profile and the possibility to add materised views to the log.

See here for details: https://www.confluent.io/blog/turning-the-database-inside-ou...