Re:Implementing Cosine in C from Scratch (2020)
by adgjlsfhk1 on Tuesday March 29, 2022 @05:00PM
(#30848290)
It's not just sub-normal numbers. As https://randomascii.wordpress.com/2014/10/09/intel-underesti... shows, fsin only uses 66 bits of pi, which means you have roughly no precision whenever abs(sin(x))<10^-16 which is way bigger than the biggest subnormal (10^-307 or so)
[ Parent
| on HN ]
Re:Implementing Cosine in C from Scratch (2020)
by ramshorns on Wednesday March 30, 2022 @06:03AM
(#30853243)
In that range, just returning x would be way better. Maybe even perfect actually - if x is less than 10^-16, then the error of x^3/6 is less than the machine precision for x.
Re:Implementing Cosine in C from Scratch (2020)
by adgjlsfhk1 on Wednesday March 30, 2022 @11:54AM
(#30856904)
the error isn't when x is small. it's when sin(x) is small. the problem happens for x near multiples of pi