A private network is ridiculously easy to set up. In our case, we had two switches, one public, one private. Every machine had 4 NICs. Just bind some to public addresses, some to private, plug in to appropriate switch.
Unless your traffic is extremely bursty, auto scaling is probably more trouble than it’s worth. We were serving millions of users with three dual-socket machines. And if you really need it, you can try to handle base load with dedicated servers, and bursts with AWS infra. But at <$10k/mo on AWS, you’re not getting anywhere near the horsepower I’m talking about.
You also have to include the human time spent setting up and maintaining those servers. And also the fact that you can probably use a managed service on request instead of 17 servers running constantly. Not even mentioning redundancy.
Most businesses are not technology businesses, they deliver goods or services to their customers where IT is part of their costs to deliver.
Investing in IT people to run that bare metal is a huge expense that doesn't improve your goods or services directly.
As for "where a task would floor 32 cores for 3.5 seconds"... means that your internal IT infra had to support that maximum requirement headroom 100% of the time, even though it may not be used for 99% of the time (depending on load patterns etc).
I understand it was a startup and may not have needed high-uptime ops from day 1, but a lot of organizations do, and the cost is not that dissimilar.
I've done this: built a 17 server cluster, housed in a colo, for a high compute startup producing 3D avatars from photos automatically. The hardware for the cluster, including a Federal Reserve class hardware firewall, was just over $50K, with a month to month colo bill of $600. This was in contrast to $96K per month if the same configuration were at AWS. This was 7 years ago, so adjust the prices, but you get the exponential higher expense point.