Re:Why Google Sheets ported its calculation worker from Java...
by wood-porch on Thursday June 27, 2024 @01:08PM
(#40812686)
Google sheets does do decimal math so I'm not sure what you mean
[ Parent
| on HN ]
Re:Why Google Sheets ported its calculation worker from Java...
by PaulHoule on Thursday June 27, 2024 @01:23PM
(#40812835)
It doesn’t use ordinary floats?
Re:Why Google Sheets ported its calculation worker from Java...
by wood-porch on Thursday June 27, 2024 @03:56PM
(#40814417)
Unless my test methodology was flawed:
Displays 0.3:
"=0.1+0.2"
Displays 3E+79:
"=(0.1+0.2)*1e80"
Re:Why Google Sheets ported its calculation worker from Java...
by csande17 on Thursday June 27, 2024 @08:07PM
(#40816530)
Try "=((0.1+0.2)-0.3)*1e80" -- your test can be fooled by pretty standard rounding for display.
Re:Why Google Sheets ported its calculation worker from Java...
by wood-porch on Thursday July 04, 2024 @02:53PM
(#40877159)
Oops -- nice, thank you! Sorry, guess I was mistaken
Re:Why Google Sheets ported its calculation worker from Java...
by mkl on Thursday June 27, 2024 @08:30PM
(#40816669)
That could be just the way it's displaying, e.g. to 14sf or something. If you try adding a million 0.1s and use a formula to compare to 100,000, that might tell you (with floats I got 100000.00000133288).
This question suggests it uses floats: https://stackoverflow.com/questions/74542790/google-sheet-yi...