I would agree though that for startups, the problem is finding the place to $$$, it is important to know what the cost per user/connection/instance is and being able to control or monitor it.
It's often a lot easier to rely on horizontal scaling, especially on a strategic level when you're deciding whether to trust your organization's technical ability (which really means management's hiring ability) or your ability to call AWS and spin up another hundred servers.
"Easier" doesn't mean "cheaper" or "faster" in the long run, but it is lower-variance for sure.
In other words, horizontal scalability means being _able_ to throw hardware at the problem :-)
Anyway, I agree with fiatmoney's sentiment that if you're in a corporate environment where your goal is growth, it's often best to focus on ensuring that your systems can grow along with demand, in a confident, easy, operations-free way -- as opposed to ensuring you've eked every bit of performance out of the hardware you have. In many business projects, the cost of human engineering time exceeds the cost of hardware.
Even when considering large-scale systems, it makes sense to scrutinize optimizations. If you had time to implement only one of these two equal-effort projects, which would you choose? (a) An optimization project expected to reduce a recurring yearly cost by $1 million (b) A new feature development project expected to earn an additional $2 million yearly recurring revenue?
I've found it effective to look at all engineering decisions as business decisions. From a business perspective, the mistake at the center of fallacies like premature optimization is the failure to consider opportunity cost.
(On the other hand sometimes pushing a technical project as far as you can is its own reward. Not everything in life has to be about business. Whether or not the results are widely relevant to typical businesses, I look forward to seeing the research into the C10K and C10M problems!)
Impossible to answer without knowing the gross margin of your business. For a big-box retailer (low margins), take the first. For a high-(gross)-margin business, take the second.
In modern times, RAM and CPUs are more than 10 times bigger and faster, but I am seeing people get around 25 times LESS out of them, because they choose terrible tools, don't benchmark, and generally don't care. Now (different company) our app server pool has 150 nodes and each serves 4 users. The application complexity is significantly smaller than what I did 13 years ago.
I sincerely doubt any of my coworkers have read this document. It shows.