This must be common or we worked at the same company, seen this exact pattern.

It went like:

App -> DynamoDB -> Kafka Connect Sink Process -> RDS -> Kafka

The reason for all the middle processes were because teams couldn't agree how to structure their data and the first app would dump literal nonsense sometimes so the Kafka connect process's job was to clean it and dump any of the nonsense they pumped into it. Pretty sure there was a gnarly log aggregation layer in the middle somewhere too IIRC.

I've seen so many people spend weeks if not months "working" to avoid doing a trivial database migration. Database fear is overwhelmingly powerful in a lot of people it seems.

> Database fear is overwhelmingly powerful in a lot of people it seems.

Database are still fairly poorly documented when it comes to administrative work.

There is an incredible amount of tutorial, books and courses on how to write sql queries and stuff... But there is almost zero content on how to properly administer a database.

I mean, from novice admin to DBA-level capabilities.

I said all this before and i'm ready to write this again: i think there's a good market space for dba-style courses.

This does not honestly seem true for the main sql databases. For everything else, yes, but if people were actually learning databases instead of hiding from them most of the uptake there wouldn’t exist anyway.

But, going the other way, I worked for over a decade on Goldman Sach's SecDB system. It's a quirky steampunk alternative future that branched from our light cone around 1995. There's a globally distributed eventually consistent NoSQL database tightly integrated with a data-flow gradually-typed scripting language (and a very 1990s feel 16 color IDE).

There's a service that watches the transaction log of your regional replica so that you can make long-poll HTTP requests that return when any change matching your filter is committed.

It makes a lot of sense for a financial risk system, where you end up calculating millions of slight variations on a scenario. The data flow model with aggressive memoization makes this sort of thing much cheaper.

However, I saw plenty of systems written where you'd attempt to write your request to the next key matching some regex (and retry with the next key if it already existed), where your request would contain some parameters and the database key and/or filesystem path where results should be written.

Message queueing triggers a host of psychological needs. Synchronous jobs rarely need monitoring and management features, but move the same work to a queue, and everybody loses their minds.

I think I've seen enough complexity created by engineering teams given total autonomy, with hands-off leadership, that I'd prefer a much more constrained approach. There should still be autonomy, of course, but proposals for new tech, languages and paradigms should only be considered with due diligence.

The most unpleasant codebases I've dealt with are ones that have suffered from a lack of strong leadership, and they are almost uniquely microservice setups that pull in everything but the kitchen sink, usually because it's just trendy to use it. Monoliths can get pretty damn ugly too but at least it's contained in one single codebase.