Re:The many ways of converting FP32 to FP16
by corsix on Monday April 24, 2023 @04:13AM
(#35684924)
Assuming that you're after "round to nearest with ties toward even", then the quoted numpy code gets very close to `vcvtps2ph`, and one minor tweak gets it to bitwise identical: replace `ret += (ret == 0x7c00u)` with `ret |= 0x200`. Alternatively, the quoted Maratyszcza code gets to the same place if you replace `& 0x7c00u` with `& 0x7dffu`.
[ Parent
| on HN ]
There are no comments on this comment yet.