News for hackers. Stuff that matters.

A comment on 3D engine entirely made of MS Excel formulae. Parent comment

Re:3D engine entirely made of MS Excel formulae by eggy on Sunday September 23, 2018 @02:29AM (#18049693)
A lot of Excel's issues are well-known because of how pervasive it is. Python and kin rely on C libraries and other layers of imported packages for numerical work that can lead to all sorts of numerical mischief. Even doing:

  round (2.575, 2)
in Python leads to 2.57 instead of rounding up to 2.58. Everything in our engineering firm is checked by another engineer.
[ Parent | on HN ]
Re:3D engine entirely made of MS Excel formulae by zokier on Sunday September 23, 2018 @11:12AM (#18051402) 

    In [7]: round(2.575, 2)
    Out[7]: 2.58
Umm?
[ Parent | on HN ]
Re:3D engine entirely made of MS Excel formulae by EForEndeavour on Sunday September 23, 2018 @02:05PM (#18052314)
From the Python 3.7 docs (https://docs.python.org/3/library/functions.html#round):

The behavior of round() for floats can be surprising: for example, round(2.675, 2) gives 2.67 instead of the expected 2.68. This is not a bug: it’s a result of the fact that most decimal fractions can’t be represented exactly as a float.

[ Parent | on HN ]
Re:3D engine entirely made of MS Excel formulae by zokier on Sunday September 23, 2018 @04:14PM (#18052920)
Well that is just normal float fun :)

    In [17]: Decimal(2.675)
    Out[17]: Decimal('2.67499999999999982236431605997495353221893310546875')
So sort of round() is working as expected, but the number you are not inputting is not the one you are expecting.
[ Parent | on HN ]