Re:One does not simply calculate the absolute value
by im3w1l on Sunday August 22, 2021 @05:06PM
(#28269599)
NaN values can have sign bit set. This will unset it.
[ Parent
| on HN ]
Re:One does not simply calculate the absolute value
by im3w1l on Sunday August 22, 2021 @08:01PM
(#28270918)
Too late to edit now, but the floating point standard actually specifies that this is what abs should do.
Re:One does not simply calculate the absolute value
by gene91 on Sunday August 22, 2021 @05:29PM
(#28269809)
You made a good point. But one might argue that -nan should become nan after an abs operation. I don't know enough about this topic to determine whether such an argument has merit though.
Re:One does not simply calculate the absolute value
by HWR_14 on Sunday August 22, 2021 @06:43PM
(#28270430)
It's not -NaN vs NaN. There are a bunch of NaN types, which are not related like that. It could (according to the spec) change "NaN: Attempt to arcsin value greater than 1" to "NaN: Zero divided by Zero operation"