Scaling is a hard problem. I've worked at places where we used a whole network of auto-scaleable services and guess what - you will still have problems. Each managed service has tradeoffs, often ones that you only encounter once you'd made a substantial commitment to that service. There is no free lunch and you're fooling yourself if you think there is one.

Often you get lucky and the cluster of managed services you select happen to scale along the metrics of your resource use. Many people's resource use patterns are similar and the managed service people take advantage of that. This is nice! But it's a trade for downside risk: you may find that your resource use patterns differ from the 90% case and your spend goes up very fast or your scaling hits a wall.

In my experience a lot of designing a service architecture is picking where you want your complexity. Services (in or out of containers) running on VMs have a simple billing and architectural model. In my experience they form a good basis to organize your other resources around and are a good foundation to grow from.

I think this is a joke people missed. Surely scaling multi-dozen-users-at-the-same-time was obvious enough that people didn't start talk about scaling seriously, but that's what happened.

If you app has multi-dozen users, and needs to "scale" something is wrong.

Yeah, my joke didn't scale well. We did well with servers (sans cloud) and comolex apps before.. something changed in collective mindset in the meantime.

How do you scale your cloud app?

Usually, the hard part of scaling isn't raw compute power, it's scaling your datastore after you've already exhausted the option of throwing more compute power at it, and this problem remains whether you're on the cloud or not.

Until you hit that problem however, throwing more hardware at it is the right solution (you may be surprised just how much load a single Postgres server on bare-metal can handle).

I agree with this in general, but would caveat that AWS etc. have made throwing more hardware at the datastore solution a lot easier. You're right that a bare-metal Postgres monster can serve a lot--but Aurora Serverless V2, if you can live with its (pretty mild IMO) quirks and if you can pay for it, is a profoundly hard-to-argue-with offering.

Buy more servers, loadbalance, and don't take the short painful route architecture-wise?

It isn't really that hard.

Can't wait to end my next system design with "just buy more servers. scaling isn't really that hard."

I've done exactly this in many meetings. It's a balance between infra costs and dev time optimizing spend. Typically ping pongs back and forth.

There is a point where that becomes exponentially prohibitively expensive.

At a former startup where I worked as senior engineer, that was our original approach. Then one weekend we tripled our userbase and horizontally scaling required massive changes in database architecture, sharding solutions, etc.

"Just buy more servers and load balance" is the short painful route. Carefully planning out and taking advantage of scalable architecture that can be provided to you less expensively because it too, runs at scale is the hard method. The fact that its easy to shoot yourself in the foot with it doesn't make it the easy route.

>Then one weekend we tripled our userbase and horizontally scaling required massive changes in database architecture, sharding solutions, etc.

You took the short easy route out of the gate instead of building for what you were planning to handle. This is the opposite of what I espouse. If you were going about it my way, you build that in from the ground up. Harder and a bit bumpier to roll out from the get go, but has worked well for me in the past.

If you want to spend capital for an unpredictable eventuality that may never pass over taking advantage of another company's investment to scale cost... sure, that's definitely the hard route.

I wouldn't call it "good" or "intelligent", but it's definitely hard.