> Why do you think that third situation unlikely?
Using a non-Intel architecture for general-purpose compute slices seems like, at minimum, a huge toolchain investment.Things are probably better now than they were five years ago (eg there seems to be LLVM support), but even assuming toolchain parity we're talking about having two flavors of each binary to maintain; a new constraint to consider when scheduling jobs; etc. (Edit: The HHVM ARM target is interesting, and mitigates some of these).
It's not impossible, but IMO pretty unlikely compared to the other more "traditional" possibilities.
You're definitely right there is a large software maintenance cost, but that has to be weighed against any perf/watt savings (Anandtech's Calxeda review shows even early systems have potential http://www.anandtech.com/show/6757/calxedas-arm-server-teste...), being able to put pricing pressure on Intel, and not being tied to Intel as a single supply source.
There is quite a lot of assembly code in open source stuff, there was a survey here for the arm64 porting https://wiki.linaro.org/LEG/Engineering/OPTIM/Assembly although much should be removed... (the 64 bit port could be what Facebook is interested in and thats way less supported...)
[0]: http://h17007.www1.hp.com/us/en/enterprise/servers/products/... [1]: https://cards.linaro.org/browse/CARD-327