It makes no sense for a single team to run 2000 microservices that come together into a single app. The amount of overhead for managing so many interfaces is insane.
At the same time, it's hard to justify 2000 developers working on a single binary. You end up with entire teams dedicated to managing and deploying. Companies do it (Google), but it's not without costs.
If every microservice runs with the same specs (container size/# of containers), then there is nothing gained from scalability. If anything, you're probably wasting a large amount of resources if your containers cannot shrink any more.
At the same time, if you are deploying thousands of copies of of a single binary, when most of the resources go to 1% of the code, then you're wasting resources with needless copies.
The (micro)services fad is definitely brought on by the recent rise of virtualization. It's probably a bit overboard.
I also get the sense that a lot of purists moulded in the ways of yore are alarmed at the waning relevance of their skillset. This to me seems like a bigger problem than the evolution of software paradigms.
My only complaint is this ostrich mentality of declaring ubiquity and ignoring the downsides and unsuitable cases. I suppose drinking the koolaid might be necessary to really buy in and push the paradigm forward, but sometimes it feels like willful ignorance.
I think the problem lies more in the PR, advocacy and conference circuits where there's an incentive and agenda to present things as silver bullets. It mirrors a greater problem in public discourse that everything has become so polarized that nuanced debate is drowned out by noisy, confident blowhards.
Back when I was still an employee, I strove to take a more intermediate tact that let me be more vocal about technical direction. What's surprising is how willingly people with authority will listen to what you have to say, once they realize you know what you're doing. I've also been fortunate to have had roles that enabled that in the first place. And now I'm trying my hand at entrepreneurship, which feels practically utopian :)
Has local dev/testing been solved for that yet, and different environments, etc?
This. What happened to "make it work then make it scale" ?
The fact that it is wrong (or rather, incomplete)
It should be "make it work with a clear path to scaling, then make it scale" - that's true with respect to both software engineering and business plans.
It's the same with "premature optimization is [the|a root of [all]] evil" - Premature optimization is a waste of time and often paints you into a corner. However, if you don't think about the bottlenecks and how to optimize them in advance, you more often than not find later, when profiling, that there are not enough hotspots, and the performance is "lost" all over the place.
And I remember an earlier project that had to be completely rewritten in the early '90s because at the time, Oracle couldn't yet scale even throwing money at it (and DB2 reportedly could - but the required Mainframe was not an option).
Simple is often best, yes. But not oblivious-to-the-future simple.
And, honestly, sometimes you've done variants of the same problem enough that you know where some of the bottlenecks are inevitably going to be - so why not design them out right at the start?
Of course, its not a holy grail. But we see that kind of thing over and over again. I think its OK; it gets developers genuinely excited to try new technology. Its only when this hype influences dangerous decisions that I'm worried about.
I totally agree that microservices can be a form of premature optimization, in particular because of the cost with today's tooling. But I think there's hope that lot of those costs will go down (both in terms of dev time and infrastructure) with things like AWS Lambda, etc.
Anyway, if devs think a little more about the interfaces, I think that will be a good thing.
Don't get me wrong, a service-oriented architecture is the only thing that scales to large companies. Once you get to dozens of engineers and millions of lines of code you will inevitably need to have an SOA because Conway's law. Also, there is a learning curve to building microservices which improved tooling really helps with.
However the thing that really grates at me is how these articles say things like:
> Services are now an everyday, every-developer way of thinking
With nary a mention of the overhead. There is no way around it, distributed systems have an irreducible complexity no matter how good your tooling and boilerplate is. You have to put in extra work to decouple everything and handle failure in a way that actually reap the benefits of the distributed system. And in the end, what these articles always gloss over is the interface between these systems. If you can easily define an interface between systems that stays relatively stable as the service evolves, then congratulations, you have a good candidate for a service with minimal overhead. But for most applications, those interfaces are shifting all the time, and there is no better tooling than multiple logical services running within one binary and build system where integration testing and validation is cheap. This is a real fucking problem people, it's not going to go away because there's a couple billion dollars worth of venture-backed startups ready to blow their cash on you in the vain and most likely misplaced hope that they are actually going to have to scale to dozens of engineers. Premature scalability is one of the worst siren songs for young engineers and we're seeing it in spades right now.