Note that .net is essentially just a VM and set of APIs. There is nothing stopping someone from building site to service massive loads. The bottlenecks occur when .net is paired up with IIS, and various asp.net controls and ViewState.
For example, we use apache with mod_mono and write our own custom controls.
Couldn't the same argument ("just a VM and a set of APIs") be made for many programming languages, Web or otherwise? I'm not criticizing what you wrote, just trying to make sure I understand what you're saying.
I just wanted to point out that ".net doesn't scale" is completely incorrect.
Gotcha. I'm just arguing (mildly, 'cause the most .NET experience I have is a two-week project a couple years ago) that possibly .NET and friends is not the best choice for a massive site such as MySpace, at least if you are not prepared/qualified to completely reinvent the wheel (as you seem to have done). Then again, Facebook's solution seems pretty ad-hoc as well. Perhaps such wheel invention is inevitable.
Yeah I see your points. I just wanted to point out that ".net" is different than asp.net and IIS. Those are part of the complete default stack and are typical bottlenecks. In a way they're designed to achive easy development/deployment over performance.
asp.net out-of-the-box will not work for a massively concurrent public facing app. However, neither will php, rails, django, any of the java frameworks. "re-invent the wheel" might be an overstatement for what you need to do, but no matter what...you're going to do some serious tweaking since every app has different needs
Note that I am not saying that PHP > .NET -- obviously on sites of this large scale, as has already been noted in the post's comments, using any technology "out of the box" is looking for trouble. What I mean to imply, however, is that perhaps for sites this big, .NET is the wrong thing. And, as the parent comment points, out, IIS certainly isn't helping things, though I don't think Apache would necessarily be a magic bullet. Craigslist, which does many megabytes of data transfer per minute, is considering switching to Lighttpd because Apache too heavyweight.
P.S. Sorry about the links -- I couldn't figure out how to link a text string to a URL. Can someone enlighten me?