It goes beyond the full quote --- the whole paper is great, showing Knuth's stature as both a thinker and a writer. If you've ever quoted the "premature optimization" line to someone, or nodded when hearing it, you'd do yourself justice by understanding the context of the quote.
http://cs.sjsu.edu/~mak/CS185C/KnuthStructuredProgrammingGoT...
Here's some more from it to entice you:
My own programming style has of course changed during the
last decade, according to the trends of the times (e.g.,
I'm not quite so tricky anymore, and I use fewer go to's),
but the major change in my style has been due to this inner
loop phenomenon. I now look with an extremely jaundiced eye
at every operation in a critical inner loop, seeking to
modify my program and data structure (as in the change from
Example 1 to Example 2) so that some of the operations can
be eliminated. The reasons for this approach are that: a)
it doesn't take long, since the inner loop is short; b) the
payoff is real; and c) I can then afford to be less
efficient in the other parts of my programs, which
therefore are more readable and more easily written and
debugged.
> Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.
Another common mistake relates to the phrase "penny wise and pound foolish", which applied to programming helps us identify those scenarios where we might waste time micro-optimizing array searches rather than choosing a hashtable (or even pre-sorting the array). Time spent optimizing your architecture will nearly always offer a bigger payoff than micro-optimizations - but once that's done, Knuth's critical 3% is worth remembering.