I think there is a difference between "premature optimization" and putting thought into what you are actually asking the computer to do when you write some code. You aren't hand unrolling loops and rewriting stuff in assembler on the first pass, you're thinking to yourself "doing it this way is going to require n! iterations, but if I restructure my internal data layout I can reduce it to n log n at the cost of only a few megabytes of memory."

The trick here is that you have to understand what each tool is doing before you use it, and that's the opposite of what computer languages are driving towards these days. There is a big push towards abstracting away the actual work, potentially so the compiler can apply optimizations under the hood, but this requires the programmer to either know tons of details about what the compiler does, or to trust it implicitly and hope they don't write code that is grossly inefficient. Or maybe what you do is grossy inefficient, but it can be automatically parallelized well so you don't notice it too much.