News for hackers. Stuff that matters.

A comment on Inventor Claims to Have Solved Floating Point Error Problem. Parent comment

Re:Inventor Claims to Have Solved Floating Point Error Problem by jovial_cavalier on Thursday January 18, 2018 @01:05PM (#16179584)
> If an operation exceeds this limit, an insufficient significant bits signal "sNaN(isb)" would be raised.

What about for binary repeating decimals like 0.3? Wouldn't it always raise that signal?

[ Parent | on HN ]
Re:Inventor Claims to Have Solved Floating Point Error Problem by oh_sigh on Thursday January 18, 2018 @01:41PM (#16179973)
Presumably if you specify that you want more units of precision than are available, then yes. But if you say you only want 1 significant digit, then it can store it.
[ Parent | on HN ]