I don't think this level of computational power can be achieved on a modern CPU, or even a GPU! But GPUs are probably the closest analog to Google's absurdly parallel architecture.
To get a GPU working at maximum performance, you either have to go OpenCL2.0 or CUDA. Compared to OpenCL1.2, OpenCL 2.0 has a better atomics model, dynamic parallelism (kernels that can launch kernels), shared memory, and tons of other features.
NVidia of course supports those features in CUDA, but NVidia's OpenCL support is stuck at 1.2. So in effect, CUDA and OpenCL are in competition with each other.
Anyway, that's the current layout of the hardware that's available to consumers. I think its reasonable to expect a graphics card in a modern machine, even Intel's weak integrated-GPUs have a parallel-computing advantage over a CPU.
So for high-parallelism tasks like audio analysis or image analysis, it only makes sense to target GPUs today.
Its "weird" in the ways that matter: there's no commodity hardware in existence that replicates what a TPU does. The only place to get TPUs is through Google's cloud services.
CPUs are basically Von Neumann Architecture. GPUs (NVidia and AMD) are basically SIMD / SIMT systems.
Google's TPU is just something dramatically different, optimized yes for Matrix Multiplication, but its not something you can buy and use offline.
I'll look into them, thanks!
But you will. The entire point is to put this in a phone, so you can distribute a trained neural net in a way that people can actually use without a desktop and $500-$4,000 GPU.
As far as I can tell, they put a microphone on your phone and then relay your voice to Google's servers for analysis.
Or Amazon's servers, in the case of the Echo.
I don't see any near-term future where Google's TPUs become widely available for consumers: be it on a phone or desktop. And I'm not aware of any product from the major hardware manufacturers that even attempt to replicate Google's TPU architecture.
NVidia and AMD are sorta going the opposite direction: they're making their GPUs more and more flexible (which will be useful in a wider variety of problems), while Google's TPUs specialize further and further into low-precision matrix multiplications.
Also, I just turned on airplane mode and google assistant recognized my voice.
So, in that regard its no more "weird" than other common accelerator/coprocessors for things like compression.
So, in the end, what would show up in a phone doesn't really look anything like a TPU. I would maybe expect a lightweight piece of matrix acceleration hardware, which due to power constraints isn't going to be able to match what a "desktop" level FPGA or GPU is capable of much less a full blown TPU.
Edit: I should say that in state of the art results there tend to be multiple components, including multiple neural nets and the tricky "decode graph" that gok and I are talking about. These are trained separately then get stuck together, as opposed to being trained in an end-to-end fashion.
The issue previously has been a lack of large enough high quality annotated datasets, and open source ASR libraries being a bit behind or not well integrated with cutting edge deep learning. I think that's changing now though. I hope it won't take too long until pre-trained, reasonable size and high accuracy TensorFlow/Kaldi models for many languages are common.
The dataset is only going to grow as time goes on. They just need to get their website in order (a stats page to show where they're currently at and how far they need to go would help a lot) and throw a little advertising behind it. It's just a matter of time
As the article says latency is still a problem and it's a huge problem in current open source solutions, some stuff I was testing was easily 5 seconds. I know that can be improved with configuration, but when dealing with libraries of 10 words or so, that's pretty bad.
I feel like anyone who is seriously interested in this space has been scooped up by all the big companies and the open source solutions have really seemed to linger because of it. It's one of the first areas I've seen where open source alternatives are really behind the proprietary solutions. Kind of bummed me out.
Here's a blog post - http://www.googblogs.com/kaldi-now-offers-tensorflow-integra...
The easiest way to deploy Kaldi is this - https://github.com/alumae/kaldi-gstreamer-server (or a docker image of the that)
Just yesterday there was a Show HN built with the https://github.com/kaldi-asr/kaldi project, emscripten-ized: https://news.ycombinator.com/item?id=15534531
Snips was mentioned here recently but I haven’t taken a look at it.
However I will say that for my company's use case a properly configured sphinx install produces better results than kaldi. However, getting to a point where you can say that was not an easy task.
Additionally, I actually believe that for most workloads that kaldi is likely better. Not ours though.
It showed promise, but I don't know if any work is being done on it.
1. speech-to-text-wavenet (https://github.com/buriburisuri/speech-to-text-wavenet)
2. kaldi(https://github.com/kaldi-asr/kaldi)
3. Speech recognition module for Python (https://github.com/Uberi/speech_recognition)
4. DeepSpeech(https://github.com/mozilla/DeepSpeech)
5. Natural Language Processing Tasks and References(https://github.com/Kyubyong/nlp_tasks)
FYI.
I’m under the impression that Google is mostly dogfooding its open source tooling for machine learning in GCP, and actually differentiates based on trained models and compute power.
https://github.com/kaldi-asr/kaldi/blob/master/egs/fisher_sw...
Kaldi hasn't been in first place on that dataset recently, but it was a few years ago.
On other more researchy datasets (eg. for distant speakers or languages other than English), the best system is often based on Kaldi.
https://www.quora.com/Is-anyone-working-on-an-open-source-ve...
Apparently, the business model for voice services in the past has been to snap up as many broad patents as possible to keep competitors at bay. I read an interview with a google engineer a couple years back claiming the same thing. They have to carefully work around a patent minefield with their own services and how these patents are holding back better voice search on mobile technology.
(edits: spelling/grammar)
Like, adding a mic button for voice search next to their main search toolbar on Firefox, and then ask for permission to use that data for research.
You can build your voice assistants and run them for free on a Raspberry Pi 3, or Android
Not to say that there aren't open source speech recognition systems, they just aren't completely usable in the way proprietary solutions are. A lot of research goes into open source speech recognition, where they are lacking is in the datasets and user experience.
Hopefully, Mozilla's Common Voice project https://voice.mozilla.org/ will be successful in producing an open dataset that everyone has access to and will spur on innovation.