How many chips sounds a lot more like a pissing contest. What if it was a giant chip with actually a bunch of chips inside? Who cares?
So I care because I want to know if I will actually see this 8x speed improvement or if this is just marketing BS.
If NVIDIA tells me a 2080 is 8x time faster than a 1080, I know that a 2080 when released will be roughly the same price as the 1080 when it was released, and so I can expect to actually see a 8x cost/perf ratio improvement (if we put aside the fact that this are always best case scenarios)
If the 8x speed increase was achieve with 4x more TPUs (and roughly 4x the cost) like it seems it was, then this is just marketing bullshit, and we should really expect a 2x improvement in cost/perf ratio not 8x.
Will be interesting to see if the difference grows further with the TPU 3.0 or Google will just take larger margins.
But, if you insist on going by the cost metric, you've already lost because you can buy nVidia GPUs and that's a lot cheaper than any of the TPU instances :)
Virtual machine pricing In order to connect to a TPU, you must provision a virtual machine (VM), which is billed separately. For details on pricing for VM instances, see Compute Engine pricing.
How much it cost you to complete that task is what matters. How it is done is here or there as long as get the precision.
We can see right now Google with their TPUs is about 1/2 the cost of using Nvidia with AWS.
https://blog.riseml.com/comparing-google-tpuv2-against-nvidi...
The 50% is a number you made up, and isn't based in any real benchmarks. For all other tasks that are not tensorflow tasks, the V100 is the only one that works.
But if we look at WaveNet we can see Google must be taking much larger margins with the TPU 2.0 versus Amazon using Nvidia.
Rolling out using a NN at 16k cycles a second and offering at a competitive price to the old way means Google TPUs have to be way more efficient than using anything from Nvidia.
It is hard to believe Google pulled it off. But if we look at WaveNet it suggests that TTS is a solved problem.
It will be how it is done for a very long time and just the NN will improve.
Nvidia honestly needs to get on their game. Google is running a 1000 mph. Iterating to the TPU 3.0 in just a year was a big surprise.
I suspect Capsule networks and dynamic routing which were invented by Google drove the TPU 3.0 but do not know.
Hope Google will share a paper now on the TPU 2.0 and their secrets.
Lucky for Nvidia they will share and Nvidia can copy. But just keeps Nvidia behind.
So WaveNet part of the Google Assistant and things like Duplex. But Nvidia silicon is just not going to make it possible. So Google does the TPUs as it just would not be possible to do at a reasonable price with Nvidia.
They are doing 16k cycles through a NN in real-time and competing against a far less compute intensive technique.
Now do not get me wrong I am long Nvidia and been for a while. Bit disapointed on the hit after an incredible earnings report.
I think they will do well from a investor marketing standpoint as really the only alternative to Google hardware.
But they have a fundamental disadvantage. They just do not have the applications like Google. They do NOT have the data to iterate like Google. It is why they appears to be 2 generations behind Google.
What is the goal of Nvidia? It is hardware for hardware sake? IMO, it should be driven top down and just do not see that happening at Nvidia. Is their goal the singularity?
Versus Google clearly wants to create the singularity and the silicon is in support. That is a very different calculus compared to Nvidia.
We can see it so strongly this week. The Google Keynote was all about AI applications. Then the TPUs are too support. Take a look at the duplex video for example. That is the focus and the silicon is what makes it possible.
But the more success for Google and presentations like Duplex this week and the buzz across the Internet helps Nvidia if the goal is investing. But from an actual technical solution it just points out the problem for Nvidia that much stronger.
Even more so with ML training. It is much more bumpy.
But you have a lot of other issues. Buying is going to hurt cash flow versus renting is a big one for small companies or startups not well funded.
Then also responsible for updating and stuck with old silicon. So for example we just got the TPU 3.0. So the cost will decrease.
But your cost of running what you buy is static. You are not be rewarded by improvements and they are happening quickly with the TPUs for example.
Just one year and we have the 3.0.
So, how much would 4k hours of 4x 1080Ti compute time cost me to rent? Or, to put another way, how many hours do I need to use to justify buy vs rent? Or, how many hours of 4x 1080Ti compute time can I rent for $6k?
But the other issue is you are stuck with the hardware. Google just put out the TPU 3.0 and you can use them without losing all your investment if you bought your own hardware.
The other is buying the hardware is much harder from a cash flow standpoint.
You provided some generic considerations, which don't apply to my situation, and I'm a fairly typical ML researcher. In fact, I know several startups doing DL research and they all have been buying hardware. Can you give me some examples where doing DL training in the cloud makes financial sense?
Also, you seem to imply that TPU is some kind of a miracle chip, even though we know very little about TPU2 and we know nothing about TPU3. It's actually pretty embarrassing that a general purpose V100 GPU is competitive with TPU2, which is an ASIC made for DL. If Nvidia ever decides to make a pure DL chip it will destroy anything Google can design. I mean, you can't be serious comparing Google to Nvidia when it comes to designing chips, right? That would be like believing that Nvidia could make a competitive search engine.
But the future is the cloud. I have fought people on this for well over a decade and thought everyone by now got it. Apparently not.
On TPU being a miracle chip not clear to me what a "miracle" chip would even be? Plus things keep improving and how could you have a static "miracle" chip?
What we do know is Google is doing WaveNet in production in real-time. We can hear the results. They are offering at a competitive price to traditional TTS.
We can also see the TPUs are about 1/2 the cost of using Nvidia.
Otherwise that is it. But I do not know your faith but personally I would not consider that a "miracle". But maybe just me.
Maybe Lewandowski would as I have heard he prays to a AI god or something like that.
The big difference is Google is working top down and Nvidia has to work bottom up.
So Google wants to roll out the best TTS there is to the world and at scale.
So they are going to from the top down to be able to achieve.
Clearly Google has a goal of creating the singularity and the Silicon is just a piece of the equation.
Versus I am not really sure what Nvidia goal is. Silicon is not for silicon sake. But rather a means to an end.
Plus Google having in production has the data to optimize in future versions where Nvidia just is not going to be able to.
I am long Nvidia and owned it for a bit. Was disappointed on how the stock traded after ER. Do think it will do fine as Google does all these incredible AI things. It is an alternative that will be good for them.
But for the very long term I would not hold. Google is a hold forever. Nvidia would keep on it.
BTW, would say Nvidia is now two generations behind.
Oh wow. Didn't realize you're one of those. I used to be a Kurzweil fan too, when I was 18. That phase usually passes by senior year in college. Well, I guess some people need longer to see reality. I recommend cutting back on pop tech news consumption.
Take it easy, and have a nice day!
BTW, I am old as in 50s. Have to explain to me in a little more detail what you mean by your post?
I am very curious but just not following?