These kinds of fast-twitch load shedding mechanics were not designed for elasticity -- it would not be worth the engineering investment in most cases. They were typically developed to support scale-out of data models for which uniform sharding is intrinsically impossible, requiring real-time adaptive resharding instead. If you have super-fast load shedding for extremely and unpredictably biased data distributions, you are 90% of the way to a really nice implementation of elastic capacity, just add hardware provisioning.
These setups are nice as a user, because the sharded nature of a table is (necessarily) completely transparent. You can create an empty new table and insert trillions of records without every having to manage sharding or cluster capacity as the table grows. In this sense, the fact that it is running on a cluster of discrete servers does not leak through the database abstraction presented to the user.
You literally do not. As load starts to increase, you scale up automatically. The word elastic has been used to represent this pattern in the last few generations of cloud and/or computing infra.
What's the alternative? Manually ssh into some box and crank up mysql instance by hand? like its 1996?
I was about to write out a huge example, but I figure I'll just express the core logic simply. First, it takes a chunk of time to determine that increased traffic is not just random variance. Then it takes time to allocate and provision machines. And often the traffic spikes for you and for your co-tenants are not statistically independent, so the provider struggles to allocate machines in time when it most matters.
And how much do you scale up? 10x right away? Can't do that: vastly expensive, and anyway it could be a retry storm exacerbating things. 2x and then go from there? Well, if your business is serving ads during the Superbowl break, that's not gonna work. Etc. This all starts to look increasingly absurd against the backdrop of being able to just push a button and do it yourself.
I'm not suggesting never doing autoscaling. I'm just saying that wise men don't speak in absolutes, or make architectural decisions based on toy examples. Nor am I particularly bothered about whether I'm doing things "like it's 1996" - I'm not in the fashion business, so I'm purely interested in finding the optimal solution to my problem, and I couldn't care less whether it's flaming hot or whether it's from the Iron Age.
It makes sense to get out in front of known massive load events before they hit your service. If I'm launching a new service that I expect to hit the front page of HN, I'm spinning up capacity first and asking questions later. A couple hours of running large instances or extra containers costs much less than potential lost sales from users getting timeouts.
I'm not sure what you mean. You still instantiate resources with serverless products. With an AWS Lambda function, you go to the Lambda web console, click "Create a function," and type in the code for that function (of course this can also done via the AWS API). There's no mind-reading going on. For AWS Aurora, you still go to the RDS web console, click "Create a database," choose Aurora as the database engine, etc.
Well, how do you indicate to it that you want to create a database? Does the database read your mind too? How do you indicate that you want to provision some more because there's a big event coming up?
> Scale should not come at the cost of performance is easier said than done
That's essentially my exact point, yeah. It's not a novel aspiration, and they don't contribute any kind of solution. It's like saying "computers should cost less and also be gooder".
> i think the same i will still use monolith like PostgreSQL
Yeah, I think that's probably sensible for many people, though I'd also welcome more innovation. Kleppmann's idea of 'unbundling the database' – i.e. the modern database becoming fragmented into several distinct components – is I think very promising and very probable. (Of course, your business and your production environment may not be somewhere you wish to be a hotbed of experimentation.)