Re:My Bug, My Bad: Sunk by Float
by EEGuy on Tuesday March 19, 2013 @11:11AM
(#5401333)
Had to deal with accumulating noise and unpredictable rounding in a double precision supported application.
Ended up scaling the amounts to integers, then converting those known and stable amounts to _character strings_ then doing the multiplication and division by character manipulation!
Don't laugh too hard; performance wasn't a consideration but _decimal_ accuracy was. It did the job.
Ended up scaling the amounts to integers, then converting those known and stable amounts to _character strings_ then doing the multiplication and division by character manipulation!
Don't laugh too hard; performance wasn't a consideration but _decimal_ accuracy was. It did the job.