1. Higher-ups who still believed that they could offer a platform that would be more valuable than the internet at large. Hailstorm always felt like an AOL for web services. 2. Lower down in the MS hierarchy various areas were under pressure. They had technologies built on COM and Win32 that had been thrown together quickly as a response to the growth of the web. eg. IIS and COM+. Both had problems in terms of scalability (and security as exploited later) and there is was no way to improve performance without jumping down to C++. Obviously there were a lot of other issues with these tools and a "rewrite" was needed.
Microsoft were really playing catch-up here and "copying" various things from the JVM ecosystem such as garbage collection was required. It is worth reviewing Brian Harry's note re: reference counting. A large number of developers were concerned with the implications of this change but it was for the better (for more people - there is still no perfect solution here). C# appears to the uninformed to be a mere copy of the Java language but Microsoft included features to attract some C++ developers, help the transition, and improve usability over Java. SOAP is an abomination, but it is still better than DCOM which came before.
I think Rotor was one of the most valuable things that Microsoft produced in the early years of .NET. Ted Neward et al wrote a book on it with O'Reilly and an unpublished 2nd version is at http://callvirt.net/blog/post/SSCLI-20-Patch-for-VS-2010.asp.... You may not want to look at Rotor if you contribute to Mono-based projects.
Search engines were a little dumber back then :)
You have to understand that Microsoft had spent most of the 90s being clueless about the internet. Then actively trying to kill it. But by 2001, their strategy was to embrace and extend internet protocols and technologies so they would be forever tied to Microsoft.
My impression was that .NET was a marketing term for a number of unrelated technologies that all served this overarching goal, that all came out roughly around 2001. They were saying to big corporations, that your choice wasn't Microsoft versus the internet, Microsoft WAS the internet. Only better.
C# and the CLR were there to counter Java and the JVM.
Their concept of Web Services was to make web technologies a mere transport layer for an RPC protocol called SOAP, which you would build with Microsoft IDEs.
Hailstorm and Passport were an attempt to make Microsoft the centralized broker for all sensitive information on the internet. Think Facebook Connect, only imagine it was more corporate and business focused, and knew your credit card number.
Anyway, Hailstorm/Passport and Web Services are dead, but The ".NET Framework" lives on in the CLR and C#, which were actually rather good even apart from Microsoft's strategizing.