Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by modeless on Monday May 18, 2015 @03:16AM
(#9563150)
Processing half float data on the CPU will be tricky without hardware support. There's plenty of CPU processing needed for deep learning, for capturing and preparing datasets.
Half precision floating point is really something that should have been in hardware a long time ago. There are other applications besides deep learning that could benefit from the dynamic range of floating point, but don't need 32 bits. For example, imaging and audio.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by minthd on Monday May 18, 2015 @08:15AM
(#9563950)
Logarithmic number systems could offer even greater improvement(100x over gpu), if you're willing to live with more errors and/or invest some time in fixing them:
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by DennisP on Monday May 18, 2015 @09:05AM
(#9564162)
Seems like a great idea, and the last couple pages sound like they were making good progress toward commercializing it. But the document's from 2010. I wonder if they're still working on it.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by monk_e_boy on Monday May 18, 2015 @04:38AM
(#9563335)
Haven't read the article, but wanted to ask a question: Once the data is mined and the lessons learnt, isn't that it? You don't need the massive learning algorithm, just the little 'rules reading' algorithm?
Is this a thing? Something that takes a convolutional neural network and spits out a little app?
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by nl on Monday May 18, 2015 @07:15AM
(#9563749)
If I understand what you are talking about, then yes.
For example, offline speech recognition in Android phones uses a CNN that was trained on Google's GPU cluster[1]
[1] I can't find the good reference of this, but Slide 31 of this is outdated, but kind of says it (you aren't training a 2.7M parameter CNN on a phone). http://www.cs.nyu.edu/~eugenew/asr13/lecture_14.pdf
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by derefr on Monday May 18, 2015 @05:41AM
(#9563497)
Most of the interesting things neural networks do involve online learning.
For a simple non-NN analogy, think of spam detection: you want to be able to correct misclassifications, such that it won't make them again. This requires more effort than it sounds: it's not enough to just add the one piece of weighted evidence to the corpus and re-run the algorithm, because that won't necessarily make the filter spit out the new correct answer in the case of the original. Just like there's overfitting, there's also underfitting, and the naive training method results in underfitting.
Thus, what you tend to want is something more like a garbage collector: a process constantly running in a background thread, gradually retraining the system. At any given point, the system will answer questions using an MVCC-like point-in-time view of its beliefs, while those beliefs are getting played with and re-evaluated elsewhere.
Also, every time the NN changes its mind, there will be a subset of non-training samples it has already seen, that it will now classify differently than it originally did when it saw them. Continuously going back and amending its judgements on these is usually helpful.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by soup10 on Monday May 18, 2015 @07:09AM
(#9563731)
I think it depends on the algorithm. Something like siri trains on a larger data set with better hardware and uses a reduced version on the phone hardware. I think most neural networks can be reduced and compressed into a 'read only' format with limited or no new training, but it's a secondary problem.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by monk_e_boy on Monday May 18, 2015 @04:44AM
(#9563356)
The memory chips have been moved closer to the GPU, from inches to mm away (aside: inches vs mm? Imperial vs metric? What is this craziness? Have they taken a leaf out of the plywood industry where everything is measured width and height in feet and inches and depth in mm... I mean, what the flipping heck?)
Where was I? Oh, yes, moving a memory chip a few cm makes the data bus faster? I thought that the speed of light (or electrons in this case) was so quick that a few cm wouldn't make any difference?
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by modeless on Monday May 18, 2015 @05:06AM
(#9563416)
1. The speed of electrons in wires is quite slow. Electrical signals travel much faster than electrons.
2. Light travels ~30 cm per 1 GHz clock cycle. That's approaching the point where we have to worry about it, but...
3. The speed of light constrains latency, not bandwidth. GPUs care much more about bandwidth. The advantage of using shorter wires is it allows increasing bandwidth more easily.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by minthd on Monday May 18, 2015 @08:25AM
(#9563985)
Very generally ,both bus clock speed and bus power(per bit) are proportional to the parasitic capacitor that's formed between any two bus wires which is proportional to their length.
This is because every new bit on the bus needs to charge this parasitic capacitor and this takes time and power.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by unwind on Monday May 18, 2015 @05:23AM
(#9563466)
I'm not an electrical engineer, but I think that one difference can be in the required drive voltage and power for the bus. Remember that the GPU needs to drive the bus, i.e. it must apply sufficient voltage to make the lines reach the required voltages, and do so quickly. This can become easier if you know the distance is smaller; the driver can be faster since the losses in the line will be smaller too (and less risk of it picking up interference).
It's probably a differential drive which makes things more complicated in the details, but I think in general the above is true anyway.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by pjc50 on Monday May 18, 2015 @07:06AM
(#9563721)
Component pin spacing is nearly always in decimal fractions of an inch. As others have commented, it improves latency at GHz speeds. The optimal approach for this is stacked dies, like the Raspberry Pi, although there it's done for cost and area reasons instead.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by mdda on Monday May 18, 2015 @09:44AM
(#9564346)
I was at an Nvidia presentation where they made a big deal about just being 'on-silicon' rather than having to go through copper connects (to a bus external to the chip). Each join, corner, etc adds to capacitance on the wire, which then leads to delays (alternatively power consumption, and heat).
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by web007 on Monday May 18, 2015 @01:02AM
(#9562929)
I doubt I'm the only one to confuse the Pascal architecture (successor to Maxwell) with the Pascal language.
This will make all of the stuff in Caffe (C++?), Torch (C) and Theano (Python) faster by inclusion of cuDNN, a low-level CUDA-optimized library of deep neural network primitives.
also: I was under the impression single precision was fine for most deep learning applications and double precision doesen't even have good support in most libraries but I guess it depends on the use case.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by mdda on Monday May 18, 2015 @09:47AM
(#9564368)
FLOP-wise that makes sense. But for deep learning, the big deal is in the 12Gb GPU-local memory, which has enormous bandwidth (and can store more of your dataset / parameters at once). The largest concern with GPU processing is keeping the GPU adequately fed with data - and avoiding round-trips of blobs of data with the CPU helps a lot.
Re:Nvidia Pascal GPU Architecture to Provide 10X Speedup fo...
by johanneskanybal on Monday May 18, 2015 @10:29AM
(#9564626)
Oh I agree and the article talks plenty about that topic as well. For me the temptation with Titan X is primarily the "laziness" of a) not manualy having to try to parallelize AWS units and b) not needing to try to squeeze in models into 4-6gb.
Half precision floating point is really something that should have been in hardware a long time ago. There are other applications besides deep learning that could benefit from the dynamic range of floating point, but don't need 32 bits. For example, imaging and audio.
http://www.gwern.net/docs/2010-bates.pdf
But it still doesn't mean it can't be implemented outside the u.s. or in FPGA's.
Even a simulation of this in software could be really interesting.
It might be not enough if you have a lot of filters and doesn't want to care about their ordering/gains etc
If you have something like 16bit exponent and 48bit mantissa (64-bit) that's absurd
Is this a thing? Something that takes a convolutional neural network and spits out a little app?
For example, offline speech recognition in Android phones uses a CNN that was trained on Google's GPU cluster[1]
[1] I can't find the good reference of this, but Slide 31 of this is outdated, but kind of says it (you aren't training a 2.7M parameter CNN on a phone). http://www.cs.nyu.edu/~eugenew/asr13/lecture_14.pdf
For a simple non-NN analogy, think of spam detection: you want to be able to correct misclassifications, such that it won't make them again. This requires more effort than it sounds: it's not enough to just add the one piece of weighted evidence to the corpus and re-run the algorithm, because that won't necessarily make the filter spit out the new correct answer in the case of the original. Just like there's overfitting, there's also underfitting, and the naive training method results in underfitting.
Thus, what you tend to want is something more like a garbage collector: a process constantly running in a background thread, gradually retraining the system. At any given point, the system will answer questions using an MVCC-like point-in-time view of its beliefs, while those beliefs are getting played with and re-evaluated elsewhere.
Also, every time the NN changes its mind, there will be a subset of non-training samples it has already seen, that it will now classify differently than it originally did when it saw them. Continuously going back and amending its judgements on these is usually helpful.
Where was I? Oh, yes, moving a memory chip a few cm makes the data bus faster? I thought that the speed of light (or electrons in this case) was so quick that a few cm wouldn't make any difference?
2. Light travels ~30 cm per 1 GHz clock cycle. That's approaching the point where we have to worry about it, but...
3. The speed of light constrains latency, not bandwidth. GPUs care much more about bandwidth. The advantage of using shorter wires is it allows increasing bandwidth more easily.
This is because every new bit on the bus needs to charge this parasitic capacitor and this takes time and power.
It's probably a differential drive which makes things more complicated in the details, but I think in general the above is true anyway.
This will make all of the stuff in Caffe (C++?), Torch (C) and Theano (Python) faster by inclusion of cuDNN, a low-level CUDA-optimized library of deep neural network primitives.
https://timdettmers.wordpress.com/2014/08/14/which-gpu-for-d...
tl;dr: GTX Titan X = 0.35 GTX 680 = 0.35 AWS GPU instance (g2.2 and g2.8) = 0.33 GTX 960
GTX Titan X = 0.66 GTX 980 = 0.6 GTX 970 = 0.5 GTX Titan = 0.40 GTX 580
also: I was under the impression single precision was fine for most deep learning applications and double precision doesen't even have good support in most libraries but I guess it depends on the use case.
Rather than a speedup factor of 2-3.