Inherited a complete mess of a system (response time measured in minutes) running on an Xeon E3 w/8GB of RAM and 500GB of spinny rust.
It's going to take months if not years to unscrew some of the mess so I suggested to the boss the quickest way of improving it was to upgrade the server so we ordered a Xeon E5 dual processor system with 32GB of DDR4 and 1TB DC grade SSD.
Once I've ported the system from PHP5 to 7 on that and moved to a recent MySQL version we'll get a very nice perf improvement for basically free and then I can refactor the mess and at the end we'll have a better optimised system running on much better hardware.
Hardware is cheap, people's time isn't.
Sadly it's not always that simple.
People's time, salaries, falls into the base expenditure.
Hardware acquisition is a discretionary cost, which is usually squeezed particularly in a cost-centre like IT support. Or the existing hardware is under a multi-year lease from a vendor like Dell or HP and has to be paid-for whether it's obsolete or not.
Where I've worked ( big non-IT-orientated companies ) it's more typical to throw a couple of people onto a project to debug and refactor an application than buy better hardware. Or even worse, implement a new project on existing shared hardware.
(Lots of handwaving in the above, main takeaway is that it's the expected and rational path in a world where engineers are expensive and scarce and servers are cheap)