A Fast Excel Formula Parser and Evaluator

Re:A Fast Excel Formula Parser and Evaluator by SloopJon on Sunday May 15, 2022 @05:47PM (#31391468)

I've always been fuzzy on Excel's precision. It's frequently described as preserving or storing fifteen significant figures. I wasn't sure if it rounds before storing, or if that's just how it displays. After some experimentation, I've satisfied myself that it can store more than it can display.

For example, =PI() displays at most 3.14159265358979, no matter how many digits I ask it to display. If I add 1.999E-15, I get 3.1415926535898. If I type 3.14159265358979 (omitting the invisible trailing 3), then add 1.999E-15, I get 3.14159265358979.

In other words, Excel uses double-precision floating point, but never displays more than fifteen significant figures. You may be in for a surprise if, say, you round trip a spreadsheet through CSV.

[ Parent | on HN ]

Re:A Fast Excel Formula Parser and Evaluator by bitmedley on Sunday May 15, 2022 @08:43PM (#31392642)

Excel uses IEEE 754 double precision which stores up to 17 significant digits but the Excel UI itself only displays up to 15 significant digits.

[0] https://en.wikipedia.org/wiki/Double-precision_floating-poin.... [1] https://www.mrexcel.com/excel-tips/17-or-15-digits-of-precis... [2] https://www.youtube.com/watch?v=WZfjmbEDbfI

[ Parent | on HN ]