Testing's critical. But there's multiple levels of testing, as far as websites go. At the very most basic level, there's things like Google's A/B optimizer. At the other end, there's things like Amazon's obidos - a tightly-integrated testing/recommendation engine.

It's going to come to the point where if you AREN'T live testing on some level, you're going to flounder and fall beneath competitors that are. And the better your testing, the better you'll do.

(Disclaimer - I work for a Sitebrand, company that tries to offer a middle-ground in this area. We do personalization, similar to what Amazon offers, and A/B testing of various content. My opinions are very influenced by what I've seen, which is that companies that sign on with us see significant growth because they're putting in the work, the testing, and the personalization.)