99% of your code doesn't need optimization, period. If it's not inside at least two loops it almost certainly doesn't meaningfully contribute to the total runtime.

Except the one case where you have a slow approach diffused over a whole application, when all of the code needs an optimization.

That case is easy to hit with compilers or annotation and such features. It is still one thing to optimize but impact is typically big.

That's only true for application code. If you write libraries, you can't know how your library functions will be called and have to assume that most of them could end up in performance-critical code paths.