Big cloud computing players will almost certainly move this way. Actually surprised we haven't seen more of this already. But TensorFlow and the Tensor Processing Unit were smart moves by Google and I think this is just a preview of many, many things to come.
I think IBM probably belongs on the list of top contenders to make a move. Between SoftLayer, IBM's hardware and chip design and fab partnerships, their cloud/saas businesses, and their Enterprise penetration, they're well-positioned to lead here.
I think that you're right as well that all these folks will push to create developer-friendly services and APIs backed by hardware. However, I think the real wins will come from capturing market share in the Enterprise sooner than later. If I were them and I had to prioritize between the two, I'd target Enterprise customers first and foremost. You can sell services to help them migrate and that helps you get out to market more quickly than creating a fully robust self-serve platform. Moreover, if your custom hw-backed platform can offer either better price or better performance, Enterprises will pay big bucks for it. Harder to get it right with self-serve first, but you certainly need to open up to self-serve as quickly as possible over time.
There was also a (sadly failed) initiative about ten years back in Germany to build a custom ASIC for multigrid Poisson equation solvers. I'm sure there are more examples outside of my interests.
I think this was what Google is trying with their TPU for running TensorFlow for machine learning applications.
https://cloudplatform.googleblog.com/2016/05/Google-supercha...
If we're going to see any developments in alternative architectures and hardware, it's going to happen in the cloud computing space. From my perspective, this is the only sector that is in the unique position to sell specialized computing services to customers on a mass scale and make great margins in the process.
There are many interesting and useful things you can do outside of Von Neumann architectures and Turing Complete systems that were never practical on PCs. Simple services like Reddis could probably be implemented with alternative memory architectures like content-addressable memory. Many useful algorithms in the data analytic and machine learning space can be compiled down the specialized hardware using using a combination of transistors and memristors.
For companies like Amazon, Microsoft + Google, there's actually a financial incentive to harmonize hardware and software much in the same way Seymour Cray harmonized the software and hardware of Cray's supercomputers. These cloud companies don't want to be constrained to selling commodity virtual machines. They want to sell you the next generation PaaS solution like DynamoDB, SQL Azure or Firebase so they can lock you into their cloud platform.
If there's a future where developers are writing code in special DSLs that's compiled to gates on an FPGA, the cloud computing guys will be exploring it because they have a huge incentive to get their customer's into the walled garden that yields better margins. If you can create one platform service that developers love and runs on cheaper specialized hardware, you'll be able to destroy your competitors and signal an alternative hardware arms race.