Re:In Defense of Pure 16-Bit Floating-Point Neural Networks
by tysam_and on Tuesday May 23, 2023 @02:23PM
(#36048634)
My best understanding is that architecture is predetermined, which determines the number of parameters up front?
I do think that, however, having shallower bit depths over time will require some slightly deeper networks to compensate, as a result. Sorta makes sense when you think about it a bit. :) <3 :DDDD :)
I do think that, however, having shallower bit depths over time will require some slightly deeper networks to compensate, as a result. Sorta makes sense when you think about it a bit. :) <3 :DDDD :)