It's the big stuff that kills you: a snakes nest of dependencies with spooky action at a distance will never be easy to optimize or change. Focusing your design on simplicity of implementation and regularity of interface (in that order) gives you the best shot at reacting to what comes down the road later. And, as you say, it's appears to be experience that gives you the ability to recognize the simplest approach that isn't dead wrong.
http://www.jwz.org/doc/worse-is-better.html
Engineering is about making such trade-off decisions, constantly. One of the many traits of a engineers is how quickly and accurately (on hindsight) we can make them.
And back to the blogpost, I feel that the array vs hash and LINQ examples are fairly contrived. In reality, a good engineer shouldn't even consider those approaches in the first place, unless there's a really compelling reason to do so.
Totally agree. In fact the rest of his quote is "... is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."
And to add, it's important to to decouple pieces of your project so that it's easy to refactor and optimize parts as you get to them.
The OP is right that if you completely ignore performance as you code, you'll be doing things so blatantly wrong everywhere that it's difficult or impossible to fix. But, Knuth is right too: It's counterproductive to spend 10 times as long developing version 0.1 just to make sure it's the fastest 0.1 there could possibly be. This is because your early code is likely to change a great deal over the course of the project. You'll refactor, you'll add and remove features, and you'll realize you were wrong about where the bottlenecks are. Much of your early optimization effort will have been wasted.
After much programming experience, including trying both extremes, I've learned that a middle road is the most cost-effecive. The real quote from Knuth, so says Wikipedia, is: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." He's not saying to ignore performance. He's not arguing that you should blindly accept obviously unperformant architectures, such as the OP's example of iterating over a huge array when a hash would clearly be better. He's saying you shouldn't obsess over the "small efficiencies," the time-consuming rewrites of every little, rarely-called function to shave off 5% of execution time. At least not early in a project. I think Knuth would support doing that for a highly mature product, at least in some cases. Indeed, much of Knuth's most well-known work is entirely focused on performance. He's just telling people not to be overzealous.
So how does all this translate into everyday practice? I think a lot of it has to do with instincts and experience, which let you estimate the labor costs and performance benefits of individual optimizations. For example, as a web programmer, this ethos leads to decisions like this:
- I WILL prevent a given method from loading the same database records twice, even if loading them twice is the easy way. - I WON'T rewrite the SQL that my ORM generates in most cases. - I WON'T worry about the performance difference between calling an object's instance variable accessor vs reading the instance variable directly.