There's also the argument that businesses hoping to scale might be better off assuming they will need it to save themselves work down the road.

Your Data Isn't That Big https://www.chrisstucchio.com/blog/2013/hadoop_hatred.html

This is an extremely important lesson. You don't have Big Data, unless you're one of a handful of huge companies.

Years ago I joined a mid-size public company and inherited one particular service. Management was adamant that it had to migrate away from MySQL onto Cassandra due to data volume. But there were no metrics being tracked, it was driven by "we have to do it because everyone is doing it".

I started tracking actual usage, growth and performance and it didn't seem to point to a problem that needed solving. So I halted that project (to their credit, they listened to engineering input so I was able to).

A few years later when I left, it was still perfectly fine on MySQL. Nowhere close to running out of capacity. Today it's been almost ten years and while I'm not there, what I hear is that it's still fine.

My point is that large companies that do need to scale hire many developers. FAANG likely hire north of several hundred thousand engineers alone, and according to this article[1] there are only something like 1.5 million software engineers in the US.

[1]: https://www.daxx.com/blog/development-trends/number-software...