That's not a given. It's plenty common to have the most often used features (say, sub-pages, or key processes) optimized the best, with the rarely used ones lingering at low optimization. User behavior is non-linear; small changes around pivotal values can yield great results. Over-optimizing on the other hand yields diminishing returns. Making even just your home page snappy can greatly improve conversions (or whatever is your success metric).
Other than that you are right that a bit of optimization here or there can save a lot of costs. I distinctly remember certain popular phpBB forum that, while quickly growing the userbase, somehow managed to still improve performance on rather measly hardware - and without much horizontal scaling! - getting in return genuine praise from the users, and also keeping the cost & complexity in check.
- performance is a feature
- reliability is a feature
- security is a feature
- and lastly, cost-effectiveness is a feature
If the app is to be used through wide range of regions, you want geographically close datacenters for low latencies. If the app is to be used for mission-critical purposes, and contain sensitive data, you want high assurance and privacy[0]. And lastly, the cheaper the app is to run and maintain - and that includes not just the infrastructure bill, but also availability of engineers who know the platform well[1] - the better you can develop the same app within the given budget & time.
[0] there are some legal compliance requirements for handling medical data
[1] one of the reason Windows apps are rather cheap and plentiful - there's wide availability of engineers who know the platform well. Likewise for certain popular web stacks.