Microsoft did very badly. But this kind of thing is hard to do and is not a one product or an incremental-product-improvements affair.
When Microsoft was fighting for it desktop monopoly, it did a good job of leveraging it's strengths to produce products that were "bad" by many measures but which satisfied a lot of user requirements and offered the end user good value. Surface is a logical product in this long lineage (and in answer to the parent's question, you can't create a Frankenstein product like Surface with focus group because Surface had to be a "quantum leap", an ad-hoc product of bucket chemistry). The problem is that apparently either the tablet market is not fluid enough that a product can win by just being a bucket of feature or Google already muscled into the feature-side and Microsoft isn't getting that part.
And equally, until now, Microsoft didn't actually sh-t all over their basic UI. But the approach of removing the start button subtracted more existing users than "Metro" could hope to gain.
You could frame this affair by saying that not only did MS' greed not abate over the years, it grew large enough to strangle the beast. Couldn't have happened to nicer people.
When Windows-on-ARM was demoed running full Windows 7 with all the fixings I was really interested. But then they crippled it.
There isn't anything wrong with Metro. Having a single OS across every platform is good too. But sacrificing the desktop for WinRT/Metro is the opposite of leveraging their monopoly.
If I was in charge of Microsoft, I'd put the Windows 7 start menu back in Win8. I'd make it so metro applications (and settings) could run windowed in the desktop. I'd focus on simplifying the desktop experience to make it more metro-like without forcing people to switch modes. Windows RT would run all ARM Win32 apps unrestricted.
But now with windows 8 and metro ui with winRT(/JS) it seems the problem is mainly that, simply, the are not developers who want to join the ship. I personally do not know any company or indipendent developer who is betting himself in it or even just exploring it. Anyone! And, infact, if you look at the win8 market (that is also the RT market and only way to get apps for that os) is almost empty or, if you want to see it differently, full of crap-copy-useless-apps.
At least with windows phone 8 the situation is a bit different, but here who we have to thank? Microsoft or Nokia? I bet in the second one.
I'm sure they do, and I'm sure they focus-grouped the tablets extensively.
But it's very hard to convince someone of something that they really, really don't want to be true. There's a Dilbert cartoon (http://dilbert.com/strips/comic/2012-11-06/) where the Ugly Truth is that there's nothing Microsoft can do to stop its decline into irrelevance and that producing a tablet is their attempt to hide that truth from themselves.
And maybe Microsoft's internal culture makes it hard for underlings to tell their bosses bad news, so any bad news gets a layer of amelioration and obfuscation for every layer of management.
Put all that together and its easy to see how a big failure could happen.
I'm pretty much Microsoft's target demographic with Surface: I use Office all day at work, and often need to work on the go because things pop up. I might say I want Windows on a tablet, but that's not really what I want. I want perfect office document compatibility, so I can review markups and changes on the go. I want to be able to log into my company's domain and browse my company's DMS. But that's what I really want, even if I say I want Windows on a tablet. I don't really mean I want full-blown Windows with its mouse-and-keyboard centric UI on a tablet...
In an industrial/manufacturing setting, this might work because the people doing the work isn't doing anything creative (just labour). In a research and development setting, this cannot work because a decision needs to be made using information obtained from "the trenches", and management often are not in the trenches.
This is why small startups are always more agile and moves fast, can come up with products that suite their target better than a corp. I dont think this can change, unless the corp change their method of operation from a hierarchical one to a very flat, self-managed groups of small people, and trim the fat out. This has to start with "trust", and now-a-days, trust is hard to come by.
Remember how awful Vista was? How did needing to confirm your WiFi password ever get through any kind of review process?
Also he was totally confident of success of windows 8, and with that success it was confident in the metro ui and then in the ease of switching to a rt tablet.
If you see these as a chain of events, the conclusion is really simple.
The logic was: - Windows automatically gets a shitload of installs a year (from new PC sales) - Devs will flock to the platform because of the install numbers - Users will buy Win8 tablets because of vibrant app ecosystem.
The issue is that despite the revamped start screen, win8 has not led to massive demand for modern apps. So they've stalled on step 2.
Make no mistake though, Win8 is still being sold on a lot of machines. At some point the majority of PCs will have access to the Windows store.
This is why for microsoft was really important to get marketshare in the tablet market. Now they seriously risk to be stalled in step2 for a long time cause:
1 - no one is using winRT or willing to use it
2 - no one who is using win8 use mainly the metro ui or will to have a system in which that is the main ui
3 - for reason 1 and 2 the developers are not going to spend their time in making app for that ecosystem, preferring still win32 and the desktop/web functionalities or other os more strong in the tablet market (ios and android atm).
Also the fact that windows phone is using another group of libraries and the market is totally independent in relation of winRT is another problem.
It's like for the first time microsoft failed each important decision they had to make
I mean, Microsoft has huge resources, and presumably does testing with focus groups, etc. They must test these devices with potential consumers, and it's not rocket science to do some sampling to determine how many units will sell, to get a very rough ballpark figure.
How could Microsoft mis-estimate the market on a scale of this magnitude? This isn't just a random extra camera or cellphone model that maybe doesn't sell so well. I honestly don't understand how a company this large, could get it this wrong.