Edit: An 1980 IBM 3081 could access up to 32 megabytes of main memory. A 5400-series "Harpertown" can have 24 megabytes of cache. So Modern CPU cache memory vs Mainframe main memory is not an unfair comparison. But that's just overkill the real shocker is 3081 had less bandwidth to it's registers than many modern computers have to it's local network. (It was below 40MHz and consumed 23 kilowatts of power).
PS: Feel free to post some actual numbers.
Mainframes from IBM are under constant revision and update, just like server tech.
PS: We live in an age of horrable custom software, sitting on a monstrous stack of terrible commodity software. If you spend a noticeable amount of time waiting on a modern computer system it's really just a software problem. (Wait you actually want to use XML?) But, often upgrading the hardware is really the best option.
PS: I suspect a large part of this is the mindset and experience of the people building the software. Well written Java code can be about as fast a well written COBOL code, but the average Java system is horribly inefficient.