Re:Many use cases don’t require Kafka (2019)
by longcommonname on Sunday January 31, 2021 @07:19PM
(#25984100)
That's a useful stategy, yes. But scaling your database is one of the more costly things you can scale.
Re:Many use cases don’t require Kafka (2019)
by barrkel on Sunday January 31, 2021 @07:14PM
(#25984066)
Change data capture, though. That is so so useful for tearing bits of the monolith off.
Re:Many use cases don’t require Kafka (2019)
by tacticus on Sunday January 31, 2021 @07:21PM
(#25984112)
And so painful in the short, medium and long run for anyone who has to troubleshoot or operate it.
Re:Many use cases don’t require Kafka (2019)
by longcommonname on Sunday January 31, 2021 @07:36PM
(#25984229)
Maintenance costs of any software should be taken into consideration.
I can say that we've had our troubles with kafka sure. But app teams 100% love working with it instead of having to have familiarity with all the various data sources we have.
Working with all sorts of different data but all of its kafka is much easier than all sorts of data from Many data sources.
Re:Many use cases don’t require Kafka (2019)
by tacticus on Sunday January 31, 2021 @08:57PM
(#25984755)
The app teams i've seen loving it were app teams that siloed off ops but hey maybe your use cases were not mine.
I'm seeing too many places using it in some form of RPC style system as part of some "workflow" orchestration magic system with unicorns and ponies for everyone
Or you could just give "direct" access to (a DB role that only has been granted SELECT on) a single schema within your database, entirely consisting of views. Then you've got a stable data-access API that you can iterate your data model behind, with nothing more than your DB itself.
(And you don't need to put much thought into setting it up, if you don't want to; your stable schema-API-layer can just start off as a bunch of views with 1:1 relationships to your real tables, which each just "SELECT * FROM their_underlying_table".)