Re:Essential facts about floating point calculations
by pjc50 on Monday July 06, 2015 @07:25AM
(#9838264)
Extended-precision mode is a disaster because it's confined to the FPU. If it were an 80-bit datatype it wouldn't be a problem. Because of this property, it's only predictable if you write all your math in assembler. If you write C, the generated code is very vulnerable to perturbation when the compiler changes when variables are written back to memory.
It may well give you more accurate results, but if you're doing something that's inherently numerically unstable (in our case, ASIC timing feeding into k-means clustering) it can give you different results in response to tiny code changes. Force-truncating trades accuracy for stability across changes to the software.
And obviously if you want C code that gives the same results on Intel and non-Intel hardware, you can't use Intel 80-bit FPU.
(Edit: I'll concede somewhat on "within epsilon of the right answer", because there are some examples in the Matlab case where 80 bit gets much closer to the right answer to 64-bit.)
It may well give you more accurate results, but if you're doing something that's inherently numerically unstable (in our case, ASIC timing feeding into k-means clustering) it can give you different results in response to tiny code changes. Force-truncating trades accuracy for stability across changes to the software.
And obviously if you want C code that gives the same results on Intel and non-Intel hardware, you can't use Intel 80-bit FPU.
(Edit: I'll concede somewhat on "within epsilon of the right answer", because there are some examples in the Matlab case where 80 bit gets much closer to the right answer to 64-bit.)