Here's a quite compelling manifesto from Martin Kleppmann:

https://www.confluent.io/blog/turning-the-database-inside-ou...

https://www.confluent.io/blog/apache-kafka-samza-and-the-uni...

Some very interesting architectural ideas in there. In particular, the observation that if you turn off history pruning in Kafka, you have a consistent distributed log, which can then be piped off into whatever services need to consume it. That's appealing for cases where you want an audit trail, for example.

Do most systems require that sort of thing? Absolutely not. RabbitMQ is boring tech, in the good sense; it is well understood and does its job, so IMO it's a better default option where it fits.

Why not just go to their documentation?

http://kafka.apache.org/documentation/#introduction

I can understand the product from the docs, but not necessarily why Kafka-everything seems to be approaching Hadoop-everything.

Lots of problems can be understood as a immutable log of events. Kafka is a screamingly high performance, resilient, immutable log of events.

It is kind of infectious though, once you have system implemented as Kafka-and-Samza the easiest way for them to communicate e and be communicated with is more Kafka and Samza.

I am fully on board the Kafka hype train. Choo choo.

That is useful context. Thanks!

The "Turning the database inside out" article that's linked a couple of posts up about Kafka and logs in general was the one that made me sit up and take notice and really made me 'get' Kafka and the entries stream oriented processing thing.

It raises interesting questions and I've had fun producing both arguements for and against the approach.

What are your "against" fun arguments?