> It was deeply random. It's very randomness suggested that maybe it was a physics problem: maybe it was alpha particles or cosmic rays. Maybe it was machines close to nuclear power plants. One site experiencing problems was near Fermilab. We actually mapped out failures geographically to see if they correlated to such particle sources. Nope. In desperation, a bright hardware engineer decided to measure the radioactivity of the systems themselves. Bingo! Particles! But from where? Much detailed scanning and it turned out that the packaging of the cache ram chips we were using was noticeably radioactive.
They had really good x86 hardware at a time when the post-dot.com crash wave of companies was getting started, but the sales channels were what you could charge on a credit card on their site, and VARs and the like which usually wouldn't sell to startups. Dell got a great deal of that business because they actually wanted to sell, and by the time these companies' hardware fleets got big enough for enterprise sized orders they'd long mastered how to run Dell or whatever systems to provide a reliable service.
There's always more to the (inside) story. Meaning I have no idea what's going, so should stay humble if I can't keep my mouth shut.
James Gosling's shared one theory for the downfall of Sun: radioactive packaging of the hot new UltraSparc-II chips cost the company billions.
http://nighthacks.com/jag/blog/336/index.html
I so wish Sun had survived. Jini, JavaSpaces, JXTA, grid computing... I recently had to do some AWS Lambda work (serverless & nodejs) and wanted to kill myself.
Thank you for sharing your views, theories. It's actually therapeutic.