News for hackers. Stuff that matters.
Re:Publishing with Apache Kafka at The New York Times by kod on Wednesday September 06, 2017 @05:11PM (#15187169)
Kafka doesn't have a schema per message, messages are just bytes that you can serialize / deserialize however you want to. The article refers to using protobuf for messages, which does support adding fields.

If you're equating kafka topics with the idea of a schema, you can add topics.

Re:Publishing with Apache Kafka at The New York Times by l8again on Wednesday September 06, 2017 @06:35PM (#15187685)
To add to this. Having implemented a similar log-based architecture, I would say that it is much simpler data infrastructure than having a central RDBMS considering the use case. Remember we need to deal with several consumer applications with their own respective optimization models. Postgres is a perfect choice for a given application's local datastore, while for a different application they may want to use ElasticSearch as their datastore. However, the "source of truth" remains free of any such optimization requirements. You simply save your "messages", "event", "facts", whatever you want to call it in its purest form preferably immutable, and let the consumer apps create/recreate their local datastores as they deem fit.