"Hardware is cheap" is a false dichotomy. The implication is that hardware is cheap, but development is not, and performant architectures require expensive development.
In reality, neither hardware or development are cheap, and performance does not inherently necessitate more costly development. The mantra and the dichotomy are false.
The idea is that if you're worried about performance and scaling then you'll have to find a way to distribute your load over multiple physical machines anyways.
At that point it doesn't matter so much anymore whether one of your nodes handles 5000 reqs/sec or 2500 reqs/sec. Hardware is cheap.