Re:In Defense of Pure 16-Bit Floating-Point Neural Networks
by eru on Wednesday May 24, 2023 @01:48AM
(#36054536)
Thermodynamics will dictate the performance in some sense of bit-operations per Joule sense.
The more important performance metric is not the number of bit-operations, but the quality of the neural network output.
The hypothesis is that fewer bits in your numbers give you the same or nearly the same output quality, but at drastically fewer bit operations performed, and thus less Joule spent.
The more important performance metric is not the number of bit-operations, but the quality of the neural network output.
The hypothesis is that fewer bits in your numbers give you the same or nearly the same output quality, but at drastically fewer bit operations performed, and thus less Joule spent.