News for hackers. Stuff that matters.
Re:Caffe2 adds 16 bit floating point training support on the... by maldeh on Friday May 12, 2017 @01:09AM (#14321574)
From first looks, there is little doubt that that NVIDIA's Volta architecture is a monster and will revolutionize the AI and HPC market. But the article seems to avoid quantifying how 16-point FP operations are beneficial against 32- or 64-bit FP operations in real-world usecases, or how the Caffe2 / NVIDIA architecture provides any significant boost to FP16 in particular, especially apropos to images (or why FP16 is better for image data in general).

I'm interested more in understanding why Caffe2 would outperform Theano, Tensorflow, MXNet, etc. once Volta chipsets are generally available, beyond early pre-release optimization, particularly when most of the front-runners are already leveraging / taking into account NCCL, CuDNN, NVLink, etc. When the burden of adding support for new NVIDIA primitives is so low, what gives Caffe2 an advantage beyond an ephemeral "we were partners with NVIDIA first" one-up that would last for a couple of months at most?

(Apologies in advance if this post sounds overly negative, but I am constantly evaluating the current crop of frameworks for the trade-offs they enforce on the problem space, and a definitive answer would be very helpful.)

Re:Caffe2 adds 16 bit floating point training support on the... by buildbot on Friday May 12, 2017 @04:36AM (#14322313)
16 bit FP operations seem to be more than enough for most networks - in fact for inference, google claims in their TPU paper that most network parameters can be quantized down to 8 integer bits!

There is also some work on pure binary networks, xnornet and binarynet. Low precision doesn't seem to effect these networks as long as they are trained with binary weights in the first place.

Re:Caffe2 adds 16 bit floating point training support on the... by user5994461 on Friday May 12, 2017 @07:16AM (#14322959)
>>> I'm interested more in understanding why Caffe2 would outperform [...]

A vendor-made benchmark where the vendor is outperforming other vendors.

Let's not jump to conclusion on what's really fastest.

Re:Caffe2 adds 16 bit floating point training support on the... by tachyonbeam on Friday May 12, 2017 @01:27PM (#14325904)
FP16 multipliers are smaller (in terms of silicon footprint), which means that you can fit more of them on a given silicon die, and they use less power than FP32. The other advantage is that FP16 uses half as much memory, meaning that FP16 operations require half as much memory traffic, which can lead to faster computations. It can also allow you to fit models on your GPU which have twice as many parameters.

The reason you don't typically have FP16 on CPUs is that in the CPU world, cache dominates die area and power usage. The floating point ALU component of a CPU is relatively tiny. Still, even in the CPU world, Intel recently added instructions to convert to/from FP16 before operating on it, because they know there is a memory bandwidth advantage in using a smaller floating-point format.

Re:Caffe2 adds 16 bit floating point training support on the... by deepnotderp on Friday May 12, 2017 @01:30AM (#14321632)
Caffe2 seems to be engineered for efficiency as opposed to flexibility. For example, Caffe2 is apparently Facebook's go-to choice for mobile edge device deployment for applications such as style transfer.
Re:Caffe2 adds 16 bit floating point training support on the... by sja on Friday May 12, 2017 @02:57AM (#14321964)
I mean, it uses a static graph definition instead of dynamic graphs (allows for deployment optimizations), but is there anything Caffe2-specific that would give it a leg up over Theano/TensorFlow/whatever other static graph library?
Re:Caffe2 adds 16 bit floating point training support on the... by throwaway87213 on Friday May 12, 2017 @02:45AM (#14321927)
Efficient how ? In terms of memory management ?
Re:Caffe2 adds 16 bit floating point training support on the... by eleitl on Friday May 12, 2017 @04:13AM (#14322229)
TPUs are smaller in terms of Si real estate and burn less power. Also, they can be made much faster.
Re:Caffe2 adds 16 bit floating point training support on the... by endorphone on Friday May 12, 2017 @09:08AM (#14323636)
On the topic of FP-

https://dennisforbes.ca/index.php/2017/04/11/floating-point-...

Most desktop chips offer little advantage for FP16 (at best that the source memory footprint is smaller, maybe better cache hit rates), but newer GPUs can actually blaze. If it has the magnitude/precision needs for a task, it's a big win.

Re:Caffe2 adds 16 bit floating point training support on the... by raverbashing on Friday May 12, 2017 @03:55AM (#14322170)
16fp is faster, and the increased precision is not needed in most cases

(In fact there are even some experiments with a very low number of bits - can't find the link now though)

Re:Caffe2 adds 16 bit floating point training support on the... by jackmott on Friday May 12, 2017 @08:52AM (#14323525)
16 bits lets them build something that goes much faster and/or faster per watt and neural nets don't need even 16bits usually. pretty simple!