News for hackers. Stuff that matters.
Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Thursday April 26, 2018 @10:55AM (#16932404)
"The cost performance ratio reflects this."

But the TPUs are half the cost per this article?

Plus Google does the entire stack and can better optimize the hardware versus Nvidia. So it seem Google can improve faster I would think.

If there ever was a huge advantage doing the entire stack it is with neural networks.

A perfect example is Google new speech doing 16k samples a second with a NN.

https://cloudplatform.googleblog.com/2018/03/introducing-Clo...

Do not think Google could offer this service as a competitive cost without the TPUs.

This new method is replacing the method that was far less compute intensive so to offer at a competitive price requires lowering compute cost which suspect is only possible with the TPUs.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by Nokinside on Thursday April 26, 2018 @12:16PM (#16933221)
> But the TPUs are half the cost per this article?

Exactly. Nvidia can match the performance already without 100% specialized processor. It's the just the price they need to cut by optimizing their architecture for tensor processing and reducing their profits when competition emerges.

Google is not in the business of becoming a major chip maker or competing with Nvidia head on. Putting hundreds of millions into new microarchitecture every second year eats lots of resources. They just want competitive market and the prices to go down.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by shaklee3 on Thursday April 26, 2018 @11:31AM (#16932773)
I'm not sure what you mean by google does the entire stack. Nvidia writes all of the major CUDA libraries used behind the scenes in the NN libraries, such as cuDNN, cuBLAS, etc. Nvidia can likely improve their hardware significantly faster/more efficiently than Google can because their entire business depends on it. Google has incentive for improving their TPU for internal use, but they don't make any money by selling TPU time on GCP yet.
Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by ndesaulniers on Thursday April 26, 2018 @01:06PM (#16933665)
> I'm not sure what you mean by google does the entire stack.

Consider that Google has some of the best machine learning researchers, compiler engineers, hardware engineers, and infrastructure in the business working on this.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by shaklee3 on Thursday April 26, 2018 @07:47PM (#16937105)
Huh? Machine learning and infrastructure Engineers, yes. Compiler and Hardware engineers? No. What gives you reason to believe they have a lead in either of those departments other than they have a lot of money? They're forced to use the same foundry as Nvidia, and their Hardware team is likely significantly smaller.
Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Thursday April 26, 2018 @09:10PM (#16937605)
Google been buying up AI resources well before anyone else and has the strongest and deepest team at this point.

It is why so many of the break throughs have come from Google. Great example is winning at Go almost a decade earlier than anyone thought possible.

They probably two of the strongest teams with one the Brain team and then the Deepmind team. But all the other engineers and infrastructure is first rate at Google.

Really at this point do not think the $100B cash is as important as Google already built the team and now experinced resources are far more difficult to get.

The other advantage for Google is their ability to attract the top engineers in addition.

Google just got started a lot earlier on all of this.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by shaklee3 on Thursday April 26, 2018 @09:23PM (#16937700)
Google got started a lot earlier on this? Did you read what you are saying? Nvidia has been making hardware longer than Google has been a company. No, Google does not have a better hardware team. Google has the luxury of making a device that is used for a single purpose that they control. Nvidia made a device that can be used for far more and works on commodity hardware. By the way, deepmind/alphago uses Nvidia GPUs, so that was an extremely bad example.
Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Thursday April 26, 2018 @10:25PM (#16938017)
BTW,. Deepmind now uses TPUs both for training and inference and with the results we can see why.

https://www.theverge.com/circuitbreaker/2016/5/19/11716818/g... Google reveals the mysterious custom hardware that powers AlphaGo

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Thursday April 26, 2018 @10:23PM (#16938005)
Hardware optimize for NN. Nvidia dominate focus had been graphics. Big difference which we can see the results in this article.

Plus benefits not having the baggage that Nvidia would have.

But never going to be able to use a TPU for graphics.

In the end it is about results.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by shaklee3 on Thursday April 26, 2018 @11:46PM (#16938326)
Tensor cores are hardware optimized for NN. You call it baggage, Nvidia calls it extra revenue. Because some people need double precision, and those people are willing to pay a lot of money. So the V100 continues to be the cheapest way to train and do inference on NN because you can actually amortize the server cost over time. With tpu, you pay the hourly price forever. TPU are better only in the case of NN jobs that are short in length or you don't have the capital to buy a server. Anything longer, you can buy a Titan v and come out far ahead.

By the way, the Tesla cards have no graphics output, so I'm sure why you'd say they have graphics baggage.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Friday April 27, 2018 @08:20AM (#16940216)
The problem for Nvidia is they do NOT do the entire stack. So Google has the ability to better optimize and here we are seeing those results as using TPUs is about 1/2 the price of Nvidia hardware.

Baggage is a company thing. Google really has been an AI company since in the late 90s when Larry Page was asked about using AI to improve search and he replied he was using search to make AI happen.

Ha! When you amortize you are still spending money and you saying this really bothers me and is such a problem.

Too many look at things like you do and why companies get into problems. Capitalizing is not magic.

BTW, Google is also going to be able to iterate much quicker as the AI breakthroughs happen and come out with new versions that should stay well ahead of Nvidia.

The dynamics of the chip business have changed. Use to be companies bought chips from someone and then put them in to servers and sold the servers.

The problem is the company making the chips are NOT running the chips and do not have any skin in the game or the data needed to improve.

Now we have companies like Google making the chips and also running the chips and why we see power footprint being the focus far more than the past.

We will see all the big operations including Amazon make their own chips more and more.

A perfect example if Capsule networks replacing some uses of CNNs. Google with Hinton developed the Capsule network approach and will be supporting it far faster then you will see from Nvidia.

Then there is the canonical framework for AI being TF.

All of this was theoretical advantageous for Google and now we get to see they appear to be real with the pricing of the TPUs being about half of the cost of using Nvidia.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by shaklee3 on Friday April 27, 2018 @10:29AM (#16940989)
You still haven't given a single example of what you mean by "doing the entire stack". I'm assuming that's because you don't have one?

You seemed to have completely missed why Nvidia's stock has gone up 17x in 4 years while google only 3x. The dynamics of the chip business have not changed; you are focusing on a single market, DNN, which is a small piece of the entire science/engineering community. Google made a chip that accelerates DNN. They also chose not to make an API to use that hardware with outside TF. So if you could buy a tpu and put it in your own server, it would beat the V100 in performance/watt. You can't do that, so nvidia wins, because I can buy a V100, and in 51 days the price I bought it for ($8K) has already been burned through in GCP. If you need me to do the math to help you realize that now your only recurring cost on the v100 (power) is more than 100x less than the TPU, I can do that for you. But hopefully you understand now that the TPU is for a niche market outside of google, and it will never be a large source of revenue for them at $6.50/hour.

TF is not exclusive to google. Nvidia has engineers working on TF.

Your capsule example is again extremely poor. You think google can respin an asic quicker than nvidia? Not only does history say the exact opposite, but they both use TSMC.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by ndesaulniers on Friday April 27, 2018 @04:28PM (#16944020)
> Nvidia's stock has gone up 17x in 4 years while google only 3x

Not sure the market cap or the P/E are apples to apples there.

Also:

> https://www.cnbc.com/2018/02/23/secretive-chinese-bitcoin-mi...

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Friday April 27, 2018 @10:59AM (#16941222)
Not sure why I could not reply to your post so will reply here.

Find the questioning on the entire stack just baffling with Google.

People - Google has the strongest team of AI experts in the industry by a wide margin. At NIPS this year Google had more papers excepted than anyone else. The big AI breakthroughs come from Google. They solved Go a decade earlier than anyone thought possible. Would put FB #2 with AI experts but a very distant #2.

Google miles ahead with SDC.

Plus Google is able to attract the top talent better than anyone else.

https://unsupervisedmethods.com/nips-accepted-papers-stats-2...

Applications - Search, Photos, Speech, AlphaZero, Self Driving Cars, Google now has over 4k NN in production. Nobody else even in the ball park. Hand down the leader in applications.

Infrastructure - Tensor Flow now has 98k stars on GitHub. It is the canonical AI framework in the industry and really nothing else close. CNTK is #2 with 14k stars. But Google ads about 7x per days stars.

https://github.com/tensorflow/tensorflow

Then there is Google cloud infrastructure and their other engineering talents which are well ahead of anyone.

I can go on but this is so incredibly silly. There is little question that Google is leading at every layer of the AI stack by a wide margin.

This is so silly I suspect something else going on here. We do not seem to be discussing things based on reality.

Is this about Damore?

BTW, Nvidia can only spin up what they know about. Google does not share everything but luckily they do a lot for Nvidia.

In 2018 you just have to run the infrastructure to be long term viable in the chip game.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by shaklee3 on Friday April 27, 2018 @11:20AM (#16941418)
Nobody is arguing Google is the best at AI. You are arguing that translates to Google is the best at making chips. They aren't, and there's no evidence they are. Tensorflow is open source, and Nvidia contributes. Would you be willing to place a bet that more people run tensorflow on tpu or GPU?

Edit: and there are FAR more cuda users in general than tensorflow if you're trying to compare apples and oranges.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Friday April 27, 2018 @08:49PM (#16945535)
Well we have a data point that suggests they created a better chip. But just makes sense as they do the entire stack and that gives you the info they need to build a better chip.

Look a capsule networks and dynamic routing. That potentially drives a different architecture and Google has thousands of production models to use to optimize that Nvidia just does not have.

Plus it is one company so no IP issues.

But the biggy is we can see half the cost.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by shaklee3 on Friday April 27, 2018 @09:30PM (#16945663)
No, you have a data point that says they created a chip that performed better on a single test for a single domain of work. You also have a data point that says tpu can't do any 64-bit simulations. It's not half the cost. See previous comment. It's about 100x the cost after 51 days.
Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Friday April 27, 2018 @11:50PM (#16945975)
Find the questioning on the entire stack just baffling with Google.

People - Google has the strongest team of AI experts in the industry by a wide margin. At NIPS this year Google had more papers excepted than anyone else. The big AI breakthroughs come from Google. They solved Go a decade earlier than anyone thought possible. Would put FB #2 with AI experts but a very distant #2.

Google miles ahead with SDC.

Plus Google is able to attract the top talent better than anyone else.

https://unsupervisedmethods.com/nips-accepted-papers-stats-2....

Applications - Search, Photos, Speech, AlphaZero, Self Driving Cars, Google now has over 4k NN in production. Nobody else even in the ball park. Hand down the leader in applications.

Infrastructure - Tensor Flow now has 98k stars on GitHub. It is the canonical AI framework in the industry and really nothing else close. CNTK is #2 with 14k stars. But Google ads about 7x per days stars.

https://github.com/tensorflow/tensorflow

Then there is Google cloud infrastructure and their other engineering talents which are well ahead of anyone.

I can go on but this is so incredibly silly. There is little question that Google is leading at every layer of the AI stack by a wide margin.

This is so silly I suspect something else going on here. We do not seem to be discussing things based on reality.

Is this about Damore?

BTW, Nvidia can only spin up what they know about. Google does not share everything but luckily they do a lot for Nvidia.

In 2018 you just have to run the infrastructure to be long term viable in the chip game.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Friday April 27, 2018 @11:53PM (#16945983)
Well do you have any test or seen any that indicates otherwise?

What we have here is same work and half the cost. Which makes sense as Google does all the layers of the stack and is just has a fundmental advantage to better optimize.

AI it is even more important.

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Friday April 27, 2018 @11:55PM (#16945986)
Also have no idea what 100x the cost refers to. We can see half the cost.
Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by shaklee3 on Saturday April 28, 2018 @12:07PM (#16948260)
This isn't worth going on about anymore. If you can't the cost of running a GPU in your own server versus the TPU in the cloud, then I'm not going to help you.
Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Sunday April 29, 2018 @10:30AM (#16952151)
The last thing you want is to buy your own hardware. You use the cloud as you then actually use what you need.

Google creating their own silicon which is far cheaper for them to run gets you 1/2 the cost of using Nvidia. Seems like an easy decision.

Help me? Seems like you are trying to hurt me? Just curious what drives it? Does not seem rational but driven my some emotion?

Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Thursday April 26, 2018 @11:37AM (#16932825)
Google does the applications at scale and then each layer below and a big one is TF. A great example is the recent release of the new text to speech using NN.
Re:Comparing Google’s TPUv2 against Nvidia’s V100 on ResNet-50 by jacksmith21006 on Thursday April 26, 2018 @11:39AM (#16932855)
When you use a Google service that uses the TPUs they are indirectly selling the TPUs.