All this throughput nonsense is benchmrk lies. Websites are not as a general rule highly responsive. They have huge variability in response times. Who cares if it costs a bit more to run the system if it means having a nice predictable system built for simplcity and safety from the ground up?
Any system you design will have to tackle a lot of the hard problems that kernels deal with. Now maybe you have a simpler system in mind, but it has to be drastically better for anyone to consider giving up binary compatibility.
You can't call something that's been around for less than 100 years "tried and true". I'm not saying we can't have "virtual CPUs" for tasks that can handle a performance hit. But it is becoming an unavoidable impediment. "Simple" doesn't mean "easy". It means predictable, small and undoable. The exact opposite to where we are heading.
The point here is that there are good reasons to want to use different systems together and then you end up putting in lot of effort working around the limitations of the specialized environments.
Also, you can ask a lot of people to give up binary compatibility and be okay, but asking people to give up source compatibility and change applications is a much, much harder pill to swallow. Specialized environments (like the syscall-shipping CNK on Blue Gene like jedbrown mentions) are usually seen as something to work around because they are impediments to productivity. A more robust system wouldn't be considered useful if everyone has to start from scratch.
If you're running a server application you split it into some reasonable number of tasks and allocate cores to them as needed. Nothing else runs on the system at all. If you need more cores either upgrade your hardware or you build another identical machine and talk to it over the network. If you look at the way Moore designs, he's crafting a complete artifact to solve a specific problem, not just throwing generic parts together.
You probably don't want that either because the cost of moving data between cores will be so high from both a performance and a power consumption POV. The days of free coherency between cores will eventually come to an end; you can already see evidence of that in multi-socket Intel and AMD machines (if you try to ignore the NUMA nature of memory, you swamp the link between sockets and perf tanks in a variety of apps).