There's also the overall complexity cost: by pushing complexity into dozens of libraries and thousands of applications to compensate for unnecessary inefficiency in the kernel, the system can become more complex — harder to debug and harder to optimize — than if you just bite the bullet and optimize the kernel. And even then, you don't get back all the performance you lost — on Synthesis, unbuffered reads were actually faster than buffered reads on other operating systems, because you could avoid the extra copy through the userland buffer!
Assuming it's possible to optimize the kernel that much, of course. On a Netburst Pentium IV, you're fucked, and it still remains to be seen how bad the Intel speculative execution disaster ends up being.