A great programmer is able to think about performance at every point in designing something without obsessing about it to the point where it decreases reliability, readability, or extensibility.
Having written one lousy implementation in X months does not mean that the good implementation, when starting over, will take X more months.
I have never seen a functional specification reaching more than 50% accuracy on details (most are not even close). When implementing you always discover more about the problem.
What stops you from starting over tends to be that the customers have integrated with your system according to published specs, or large roll-outs have been made already.
Of course, but one is not always implementing a totally new system that justifies "throwing one away". Often one is implementing something whose design is well-known enough that one should expect to be able to do it right the first time.
Having written one lousy implementation in X months does not mean that the good implementation, when starting over, will take X more months.
Sure, but it still takes Y more totally unnecessary months, even if Y is less than X.
If you wait until everything is mostly written, you risk finding out that a fundamental design decision is causing your performance problems.
Agreed, you should measure as soon as you can and it's best to be building and measuring, not guessing where your bottlenecks are up front.
I'm all for avoiding premature optimizations, and giving priority to short and comprehensible code over faster code. But "leaving performance considerations aside" is like discussing dictatorship and "leaving human rights issues aside".
Good programmers have good hunches about performance, better programmers pick the shortest code anyway, the best programmers measure and then maybe optimize.