http://en.wikipedia.org/wiki/MapReduce
http://labs.google.com/papers/mapreduce.html
Caching is huge too... IO is expensive, RAM access is cheap. The more you can pre-load, pre-calculate, and/or pre-sort stuff and store it in memory, the better (in terms of avoiding expensive IO anyway). Caching has it's own issues though: if you cache so aggressively that you exhaust physical ram and cause more swapping, you can actually hurt yourself. Also, you have to deal with the possibility of stale data in the cache, and determining when and how to expire and reload items in the cache. But still, caching is essential, it's just not necessarily easy.
Also, for perspective if nothing else, read the papers and stuff on SEDA (Staged Event Driven Architecture). There's still debate about how effective the SEDA approach is, but reading the discussion(s) will help you appreciate the issues involved. http://www.eecs.harvard.edu/~mdw/proj/seda/
I don't know where OP's experience level is at, but I think there's no substitute for direct experience. Running, debugging, performance testing a remote server is an art all its own.
If you don't have much industrial experience with this, I'd suggest getting hold of the source code of some site that you know scales ( http://code.reddit.com/ ) and then going on EC2 and use some performance testing tool ( http://jakarta.apache.org/jmeter/ ) to check it out.
A good place to start, if you're not already familiar with it, is High Scalability: http://highscalability.com/
Edit: book recommendations:
Scalable Internet Architectures -
http://www.amazon.com/Scalable-Internet-Architectures-Theo-S...
Linux Clustering - Building and Maintaining Linux Clusters - http://www.amazon.com/Linux-Clustering-Building-Maintaining-...
High Performance Linux Clusters - http://www.amazon.com/Performance-Clusters-OpenMosix-Nutshel...
Linux Enterprise Cluster - http://www.amazon.com/Linux-Enterprise-Cluster-Available-Com...
Java Message Service - http://www.amazon.com/Java-Message-Service-Mark-Richards/dp/...
Java Message Service API Tutorial and Reference - http://www.amazon.com/Java-Message-Service-Tutorial-Referenc...
Enterprise JMS Programming - http://www.amazon.com/Enterprise-JMS-Programming-Professiona...
Hadoop: The Definitive Guide - http://www.amazon.com/Hadoop-Definitive-Guide-Tom-White/dp/0...
Pro Hadoop - http://www.amazon.com/Pro-Hadoop-Jason-Venner/dp/1430219424/...
Wikipedia:
http://en.wikipedia.org/wiki/Shared_nothing_architecture
http://en.wikipedia.org/wiki/Shard_%28database_architecture%...
http://en.wikipedia.org/wiki/Publish/subscribe
It's important to understand the difference between vertical scaling and horizontal scaling. Horizontal is very en vogue these days, especially with commodity hardware. Why? Because you can add power incrementally without spending tons of money upfront, and without requiring a "forklift upgrade" (that is a reference to needing a forklift to bring in a new mainframe or minicomputer). This is a pretty good article on the topic:
http://www.scalingout.com/2007/10/vertical-scaling-vs-horizo...
As popular as horizontal scaling is, don't ignore the possibilities of going to bigger hardware though. It has it's own advantages, especially when you start talking about physical floor space to store servers.
Of course "cloud computing" changes some of this, both by making it cheap and easy to add VPS's to scale horizontally, or by making it possible (sometimes) to easily add more processing power, RAM, etc. to your "server." Read up on Xen, KVM, EC2, etc. for more on that whole deal.