I agree there is a balance between sane and insane efforts at caching, but I disagree that no caching is a viable choice. Apache and Wordpress with no caching on a smallish VPS will fall over really quickly. This is unacceptable when you can get huge benefits very easily by installing a widely available plugin like WP-Super Cache or WP-Total Cache. It's almost no work to get that set up, and will help you survive a mini-slashdotting. That's going to let you serve an order of magnitude more traffic than usual.

An easy solution like those plugins is such a big win for so little work, I just can't agree with "it's good enough for normal use". The last thing you want is for your blog to fail when you are getting a traffic spike.