brandtg

Kafka famously left several novels unfinished... like the log, man... Just keeps going

Leave it to HN to over-analyze a romantic anecdote...

I'm curious to know the motivation behind publicizing the project at this stage in development, as it seems like the key feature (distributed transactional storage engine) is quite far away on the road map.

Are there any design documents detailing its implementation? I checked the wiki but it didn't look like there was anything there. What alternatives were considered, and why were they abandoned?

Also, is there a concrete use case for which this system is being built? If so, what are some (publicly releasable) details about the use case, e.g. access patterns, data volume, etc.?

Some caveats about Kafka replication are discussed here: https://aphyr.com/posts/293-call-me-maybe-kafka.

It is important to include these in the "exactly-once" discussion... (hard to be exactly-once if messages are lost broker-side).

Also, great discussion from Coda Hale about "CA" systems here: http://codahale.com/you-cant-sacrifice-partition-tolerance/

Small project that may be of interest to those involved in this discussion: https://github.com/brandtg/alicia

Kind of "eventually monotonic" distributed auto-increment keys. Your mileage may vary.