News for hackers. Stuff that matters.
Re:In Defense of Pure 16-Bit Floating-Point Neural Networks by tysam_and on Tuesday May 23, 2023 @05:25AM (#36041861)
Had not known about this, thanks for sharing. This does certainly seem very interesting and applicable to the problem at hand. Large values are important to keep instead of NaNs and we generally don't need the precision in my experience.

Plus, when they're that big, they're sort of a wrecking hammer to whatever weights they touch anyways, so might as well save the precision for the cleanup steps afterwards where it really counts (at whatever number of bits works best of course) :D :)))) :D :))))

There are no comments on this comment yet.