News for hackers. Stuff that matters.
Re:Google supercharges machine learning tasks with TPU custo... by semisight on Wednesday May 18, 2016 @03:20PM (#11724942)
This is huge. If they really do offer such a perf/watt advantage, they're serious trouble for NVIDIA. Google is one of only a handful of companies with the upfront cash to make a move like this.

I hope we can at least see some white papers soon about the architecture--I wonder how programmable it is.

Re:Google supercharges machine learning tasks with TPU custo... by nickysielicki on Wednesday May 18, 2016 @03:31PM (#11725053)
There's no way Google lets this leave their datacenters. Chip fabrication is a race to the bottom at this point. [1]

Google is doubling down on hosting as a source of future revenue, and they're doing that by building an ecosystem around Tensorflow.

What I think is interesting is how weak Apple looks. Amazon has the talent and money to be able to compete with Google on this playing field. Microsoft is late, but they can, too.

Where's Apple? In the corner dreaming about mythical self-driving luxury cars?

[1]: http://spectrum.ieee.org/semiconductors/design/the-death-of-...

Re:Google supercharges machine learning tasks with TPU custo... by honkhonkpants on Wednesday May 18, 2016 @03:40PM (#11725159)
Apple designs their own CPUs. I think they'd be able to field a massively parallel FMAC chip if they thought that was a good idea.

Where Apple really looks weak is in datacenters, networking, and cloud services.

Re:Google supercharges machine learning tasks with TPU custo... by nickysielicki on Wednesday May 18, 2016 @03:53PM (#11725301)
What does the iPhone of 2021 look like?

I get the feeling from today's announcements that Google sees the 2021 version of Google Now as the selling point for their 2021 Nexus line.

I don't think Apple is preparing to compete on that.

Re:Google supercharges machine learning tasks with TPU custo... by Razengan on Wednesday May 18, 2016 @08:36PM (#11727137)
Apple's strength is in consumer (and to a lesser extent, developer) ecosystems; the cozy comfortable bubble you get when you're surrounded by everything Apple. Getting access to your stuff across multiple devices is virtually effortless and continually seamless, with almost no configuration required.

Whether that's good or not may be arguable, but it's certainly a selling point for many and I don't see Google or any other company's offerings approaching the same experience, and I suspect that's by design; they have to be more open and support all devices but that kinda dilutes everything. Apple will only get stronger in that aspect IMO.

Re:Google supercharges machine learning tasks with TPU custo... by habitue on Wednesday May 18, 2016 @08:00PM (#11727006)
I would say they're already not competing on the assistant side. Siri is considerably worse than Google Now, even though it came out first
Re:Google supercharges machine learning tasks with TPU custo... by semisight on Wednesday May 18, 2016 @04:01PM (#11725379)
I'm not sure I agree with you, long term (about the chips). I think that the value here is in the ecosystem. If Google can compete with CUDA, they'll be doing really well.
Re:Google supercharges machine learning tasks with TPU custo... by kuschku on Wednesday May 18, 2016 @05:05PM (#11725957)
> There's no way Google lets this leave their datacenters. Chip fabrication is a race to the bottom at this point. [1]

I’d hope someone somewhere steals the blueprints and posts all of them publicly online.

The whole point of patents was that companies would publish everything, but get 20 years of protection.

But by now, especially companies like Google don’t do so anymore – and everyone loses out.

EDIT: I’ll add the standard disclaimer: If you downvote, please comment why – so an actual discussion can appear, which usually is a lot more useful to everyone.

Re:Google supercharges machine learning tasks with TPU custo... by vidarh on Thursday May 19, 2016 @08:35AM (#11729619)
There's little need for anyone to steal the blueprints. It's unlikely there's anything particularly "special" there other than identifying operations in Tensorflow that take long enough and are carried out often enough and are simple enough to be worth turning into an ASIC. If there's a market for it, there will be other people designing chips for it too.
Re:Google supercharges machine learning tasks with TPU custo... by cm3 on Thursday May 19, 2016 @06:22AM (#11729149)
Same misuse happens with copyright. Both were invented to foster publishing and not creating life-long monopolies. The life times of copyright and patents must be way shorter as well. Everyone builds on something that came before. It's impossible to build a better bike if you have to test drive on a street with patent mines.

Re EDIT: Downvotes must be comment-mandatory or not allowed otherwise.

Re:Google supercharges machine learning tasks with TPU custo... by codecamper on Wednesday May 18, 2016 @03:26PM (#11725002)
I don't see any mention of offering these chips for sale. You can rent them it seems via cloud offerings & that's it.
Re:Google supercharges machine learning tasks with TPU custo... by babo on Wednesday May 18, 2016 @04:48PM (#11725811)
Sure, but that's the deal. I'll buy the latest nVidia 1080 card as soon as I can but renting these custom chips per minute would be a way better option for me.
Re:Google supercharges machine learning tasks with TPU custo... by retox on Wednesday May 18, 2016 @05:00PM (#11725920)
GPUs also have this nice side effect of being great at playing games on. Purely as a guess I'd think that the gaming market is bigger than the AI researcher market.
Re:Google supercharges machine learning tasks with TPU custo... by omarforgotpwd on Wednesday May 18, 2016 @05:47PM (#11726258)
In a future where AI is everywhere, Nvidia hopes it can sell GPUs by the hundreds and thousands to large data centers. You can make a lot more money a lot faster selling your hardware this way, and Nvidia is very interested in it judging from how much they talked about it at their recent conference.
Re:Google supercharges machine learning tasks with TPU custo... by retox on Wednesday May 18, 2016 @06:06PM (#11726391)
I would be surprised if they weren't working on their own specialised chips then, though Google have the advantage of already having the software specs to build for.
Re:Google supercharges machine learning tasks with TPU custo... by CydeWeys on Wednesday May 18, 2016 @09:20PM (#11727278)
> Purely as a guess I'd think that the gaming market is bigger than the AI researcher market.

Machine learning isn't just targeting the AI researcher market though -- it's widely used by a huge number of companies, and of course, by many of Google's most important products. I would argue that those markets combined are larger than gaming.

Re:Google supercharges machine learning tasks with TPU custo... by semisight on Wednesday May 18, 2016 @03:59PM (#11725360)
Yup, I assume they're gonna keep them in house as a competitive advantage for a time. I doubt they'll do it forever; the most valuable part of NVIDIA's CUDA is the ecosystem, and I think Google knows that.
Re:Google supercharges machine learning tasks with TPU custo... by knorker on Wednesday May 18, 2016 @04:29PM (#11725622)
I would assume that the API to use these is tensorflow.

So... just use Google's machine learning cloud thingy.

The software can build the community, where the supercharging is only available when you run it on Google cloud.

(although GPU performance isn't bad either, so you don't have to, thus community)

Re:Google supercharges machine learning tasks with TPU custo... by mtgx on Wednesday May 18, 2016 @03:36PM (#11725115)
Quantum computers, OpenPower, RISC-V, and now this - I'm really liking Google's recent focus on designing new types of chips and bringing some real competition into the chip market.
Re:Google supercharges machine learning tasks with TPU custo... by DSingularity on Wednesday May 18, 2016 @03:50PM (#11725274)
What are they doing with RISC-V?
Re:Google supercharges machine learning tasks with TPU custo... by PeCaN on Wednesday May 18, 2016 @04:06PM (#11725419)
They dumped a bunch of money into it, so presumably they're at least interested.
Re:Google supercharges machine learning tasks with TPU custo... by dharma1 on Wednesday May 18, 2016 @04:00PM (#11725367)
it's ASIC tuned for specific calculations, I'm sure it's better power consumption than general purpose GPUs. Same as crypto mining ASIC's crush GPU's in terms of power efficiency.

There isn't much data yet but I'm also guessing they probably have access to much more RAM than NVidia cards and can process much bigger data sets

Re:Google supercharges machine learning tasks with TPU custo... by agumonkey on Wednesday May 18, 2016 @03:38PM (#11725122)
I'm surprised by the perf claims. Nvidia isn't doing kids play. The graph implied they were untouchable in terms of perf...
Re:Google supercharges machine learning tasks with TPU custo... by azinman2 on Wednesday May 18, 2016 @03:40PM (#11725147)
Ndvidia has to be general purpose. This is not and thus can be better optimized.
Re:Google supercharges machine learning tasks with TPU custo... by fiatmoney on Wednesday May 18, 2016 @03:43PM (#11725198)
"General purpose" isn't that general, if you look at the actual operations they support and their threading model. It's already fairly optimized for these sorts of operations, and this amount of claimed headroom makes me suspicious.
Re:Google supercharges machine learning tasks with TPU custo... by Symmetry on Wednesday May 18, 2016 @06:12PM (#11726427)
Google has a lot of potential options that NVidia doesn't have. They can size their cache heirarchy to the task at hand. They can partition their memory space. They can drop scatter/gather. They can gang ALUs into dataflows that they know are the majority of machine learning workloads. They can partition their register file at the ISA level or maybe even drop it entirely. They can drop the parts of the IEEE754 floating point spec they don't need and they can size their numbers to the precision they need.
Re:Google supercharges machine learning tasks with TPU custo... by azinman2 on Wednesday May 18, 2016 @03:52PM (#11725283)
The fact that I can compile arbitrary programs for the GPGPU means it is general purpose. NVIDIA isn't writing softmax or backprop into silicon as a CPU instruction.

Look at how much faster ASICs for bitcoin mining are than the GPU... orders of magnitude.

Re:Google supercharges machine learning tasks with TPU custo... by fiatmoney on Wednesday May 18, 2016 @03:59PM (#11725359)
"Backprop" isn't even close to something that would be a "CPU instruction", it's an entire class of algorithm. It's like saying "calculus" should be a CPU instruction. Matrix multiplication & other operations, on the other hand, do neatly decompose into such instructions, which have been implemented by NVidia et al., since that's the core set of functionality they've been pushing for like a decade now.

Additional die space on additional functionality might hurt the power envelope (which is where the focus on performance / watt rather than performance kicks in) but it doesn't make your chips slower per se.

Re:Google supercharges machine learning tasks with TPU custo... by agumonkey on Wednesday May 18, 2016 @04:10PM (#11725449)
That was my impression too. ML under the hood was a lot of linear algebra, not very different than most shaders. But maybe Google decided to hardcode a few important ML primitives because the ROI was that good in terms of grabbing customers. Also they might have very large scale applications not found elsewhere that motivates this.
Re:Google supercharges machine learning tasks with TPU custo... by azinman2 on Wednesday May 18, 2016 @05:16PM (#11726037)
Ok I was obviously oversimplifying things but my point is since we can only speculate, it's clear that when you know specific algorithms/math operations/memory layouts/applications you want to optimize for you can create dedicated chips that optimize and do that quickly. That bitcoin miners are all dedicated chips and run circles around GPUs demonstrates exactly this fact.

Furthermore the fact that ML can be error tolerant means you also get to optimize certain floating point operations for speed or energy efficiency at the cost of accuracy. NVIDIA doesn't get to do this in their linear algebra support.

Re:Google supercharges machine learning tasks with TPU custo... by vintermann on Thursday May 19, 2016 @02:19AM (#11728385)
Bitcoin mining is an extremely well-defined task compared to machine learning. It remains to be seen how general these TPUs are in practice - whether they will support the neural network architectures common two years from now.
Re:Google supercharges machine learning tasks with TPU custo... by agumonkey on Wednesday May 18, 2016 @06:40PM (#11726609)
tbh I felt like realizing what you meant earlier at the end of my comment. I should have ps'd it.
Re:Google supercharges machine learning tasks with TPU custo... by emcq on Wednesday May 18, 2016 @10:14PM (#11727475)
If they balance compute to memory better than GPUs, you could definitely see a 10x. GPUs have large off chip memory and small caches (like 256kb). Cost to going to off chip memory can be 1-2 orders of magnitude more than on chip memory. You can certainly fit 4+MB on modern processors, but they likely bought designs from a company like Samsung because designing high performance, low power memory cells is tricky. I'm surprised they were able to keep things a secret.
Re:Google supercharges machine learning tasks with TPU custo... by honkhonkpants on Wednesday May 18, 2016 @03:47PM (#11725240)
What graph?
Re:Google supercharges machine learning tasks with TPU custo... by agumonkey on Wednesday May 18, 2016 @04:08PM (#11725438)
In the talk there was a two bar graph

    {(others, ~bottom) (google, ~top)}
Couldn't see more, but after Nvidia claiming overwhelming power with their latest GPU architecture including in the ML domain .. I was surprised.