News for hackers. Stuff that matters.

A comment on In Defense of Pure 16-Bit Floating-Point Neural Networks. Parent comment

Re:In Defense of Pure 16-Bit Floating-Point Neural Networks by eth-mld on Tuesday May 23, 2023 @02:00PM (#36048242)
https://github.com/NVIDIA/TransformerEngine - mixed precision FP8 is already here and provides similar accuracy to FP16/BF16
[ Parent | on HN ]
Re:In Defense of Pure 16-Bit Floating-Point Neural Networks by villgax on Tuesday May 23, 2023 @11:50PM (#36053853)
Yeah and for some reason limited to only hardware support of H100. Even the cost of doing it in software is outweighed by the speed & storage gains from it
[ Parent | on HN ]