From a high-level perspective, is there really any difference between DMI and HTTP/gRPC/et.al. if you could hypothetically run all of your services on a single logical node? If your entire production stack runs on your macbook, why cant you just wire it all up as a single binary and throw it on a nice powerful server? I guarantee it will run better without all that ridiculous network glue in the middle.
If I am being modest, I would say that we feel really, really stupid from where we are sitting right now. Primarily for having previously spread this implementation across 10+ services talking HTTP for absolutely no good reason. I am growing into a mindset where the entire business system needs to fit into 1 binary image. Perhaps duplicated across more than 1 physical host with differing features enabled per, but at least sourced from the same solution/repository.
What microservices allow is for different teams to use different technologies and to upgrade components independently. On service might by Python+postgresql, another Ruby, and third also Ruby but two versions ahead.
It feels fast and agile in the short-term to decouple teams like this. In the long-term, it's almost always a maintenance nightmare. You end up with way too many different technologies.
I usually break up my projects into libraries (in Python, independent pip-installable packages) with their own unit tests, and some pretty weak glue in the middle. The service layer is omitted. I design things to scale horizontally too.
I've built apps with a few services, but I've never seen a good example of a microservice architecture that beat good old-fashioned clean abstractions in libraries.
Every few years, someone comes up with a new way to do abstraction, and compares that to no abstraction (rather than good abstraction with the previous model. Microservices seems like the dumbest of the bunch I've run across in my career.
If the microservice architecture has made you less productive than the top leadership is to be blamed.
Microservices are all about scaling development agility in exchange for increased technical overhead.
In many cases this tradeoff is atrocious. If you can say "we can easily take on (large feature XXX) and deliver that", microservices are all pain and no gain.
Where they really start feeling good is when your "two week sprints" turn into about two days of available dev work because the remaining time is spent regression testing your changes and trying to keep the build from breaking because of tests you didn't touch failing (spooky action at a distance is real and it hurts). A clear API contract that both sides of the fence can develop against means that you can keep making progress.
At least until it's time to integrate against the real world, not your mock ;-)
Microservices can be a great pattern to deliver quickly _and_ maintain high availability/low technical overhead, that's their point. I think the problem arises when developers try to over-optimize or over-engineer a system, or they want to get promoted so they write a new service, ending up with huge entangled webs like in the sketch.
Individuals and interactions over processes and tools, every time
Yes – I was talking about those same monolithic architectures in that comment ;-)
As you move up in leadership, the tool that you have is organizational design. Doing it right and you have a highly efficient org. Do it wrong, and you'll constantly hit roadblocks.
How's that working out for you?
In most cases, and I'm talking right up to the scale of very large enterprises[1] you don't need that much complexity to scale technically and, if you avoid the complexity, you don't need that many people either.
And I don't think you can ever afford to be myopic if you want to effectively deliver solutions to real business problems.
[1] 90k employees, specifically, but also, specifically, not a tech company. A traditional company with a tech function.
It oversimplifies things a bit, though.
It's odd; I've never seen a "monolith" simplified by moving to microservices, but everyone seems to be doing them.