... of course, the best-paying jobs are at the places that really do need that scale, so of course we all want an excuse to play with those tools, even when it's not the right business choice. And it's not as if software folks are the only ones with a principal-agent problem, in business. Oh well.
I dislike using boring as a measure of sufficiency. It makes me wonder if those drawn to the clever solutions carry a "I know better than you" perspective, the kind of dogmatic optimism that tanked SVB.
There is ScyllaDB for a much better reimplementation of Cassandra/Dynamo, but wide-column databases are still best for niche scenarios, especially as RDBMS are rapidly evolving into natively distributed architectures.
Everything from StackOverflow to ad serving systems to high-frequency trading exchanges are run this way.
Boring to me is a load balancer, app server and cache. Once you start talking about background jobs and queues I don’t see how that’s any simpler than the alternative.
You could say Google, running Borg, is basically a database with a bunch of load balancers and background job processors. Obviously the complexity is in the scale and implementation.
With most services, until you go over a few million users concurrent users, you really don't need to horizontally scale.
So you have:
a) No enforceable isolation, reliance entirely on good behaviors and module systems
b) No independent scaling
c) No isolation of security components so instead of a minting service and an edge service you have one service that does both
d) Single language lock-in
I could go on and on an don
Serverless postgres-as-a-service isn't a stretch at all.
This thread does remind me of a previous thread, I believe it was something like optimizing a basic string operation (concatenation?) by going down to assembly and SIMD. Few people have a need to know the details of that particular implementation, and mere mortals cannot comprehend the wizardry going on there either, but that's just fine because if it gets incorporated into standard libraries or the underlying implementations that said standard libraries use, everybody benefits without ever having to understand the inner workings.
Heroku and similar things are the sweet spot right now for me. I don't have to deal with much, and I get exactly the amount of flexibility that I need. Sure it'd be a little nicer not to deal with Express on my own in a NodeJS environment, but right now it's easier to deal with than it is to avoid. Some day I'll happily go serverless.
Overall tho I agree with your statement, and I now build most of my apps using serverless and databases like Neon.
There's always instances with state somewhere; it's just a question of how smart your serverless infrastructure is about pooling the instances.
So we introduced so many optimizations just because we can't persist state. I can't help but think this is being penny wise pound foolish; the problems this article is solving wouldn't have been problems in the first place if you choose a boring architecture.