Really the big problem at this point is that stock WordPress falling over under even moderate traffic is such a common experience that having caching broken out into an optional plugin doesn't make a whole lot of sense. WP should probably ship with at least a basic caching solution built in.
Also, on ubuntu (and centos, but haven't used it in a while), it is something like 'apt-get install apache-mod-php' and everything is there :)
Not to say that your setup isn't better (it obviously is), but I want to make sure no one's intimidated by the task of hardening their WP install because they think they'll need to do both 1 and 2 for it to be worthwhile.
Since the content of my site only changes a couple times a month I started migrating away from towards a static solution. I'm not trying to say WP is bad but I think if your content is relatively static and you aren't doing a ton of updating, just make it static HTML instead of doing hacky cache stuff. It's simple, secure and runs on any type of server.
The only thing that's lacking right now is a really solid static site generator. Octopress/Jekyll are good solutions but can be a bit frustrating and not very portable.
I've never really understood the recent fascination with static site generators. I get that it's neat you can upload just everything to S3 and host it there with no servers, but then if you want to change something site wide you have to reload everything. Dynamic app + with caching reverse proxy + optional CDN is a tried and true method. I've been on projects that use this method with small VMs to power sites with tons of traffic.
1. Add memcached 2. Serve static files via nginx
It allowed me to handle 2.9 million requests
I documented it here: http://idiallo.com/blog/handling-1-million-web-request
1. Use the Wordpress supercache plugin.
2. Put Apache behind Nginx, and limit the number of Apache processes to ~6. This way you're effectively using Apache as a "PHP app server", simulating an architecture similar to Passenger/Unicorn. Nginx acts only as a buffering reverse proxy, shielding Apache from slow clients.
Why this instead of running Wordpress/PHP through Nginx using php-fpm? It's because Apache is just easier: mod_rewrite rules work automatically, etc.