Re:My Bug, My Bad: Sunk by Float
by nightowl03d on Tuesday March 19, 2013 @12:53PM
(#5402028)
I should have said a well chosen epsilon. If the computation is good to 0.001, (based on a detailed error propagation analysis of the algorithm against the expected input range), then I would use an epsilon of 1/512 when comparing against an int. The idea is that with a coarseness at that level the compiler rounding would likely have little effect. The comparison number would be (int + a power of two). But it sounds like for your scenario a scaled int or a rational would be more appropriate.
If you have access to a university library, (most universities will let you sign in and browse), is Chapter One of Stoer and Bullrish text Introduction to Numerical Analysis. (ISBN 0-387-954452)
It is on my desk because I am working out the epsilon for an algorithm a colleague designed. :-)
If you have access to a university library, (most universities will let you sign in and browse), is Chapter One of Stoer and Bullrish text Introduction to Numerical Analysis. (ISBN 0-387-954452)
It is on my desk because I am working out the epsilon for an algorithm a colleague designed. :-)