News for hackers. Stuff that matters.
Re:The tragedy of 100% code coverage (2016) by StavrosK on Tuesday May 09, 2017 @06:03AM (#14298795)
How do you test it by brute force? How is the algorithm supposed to know what the right thing to return is, unless you rewrite the whole thing, correctly this time? And, if you've done that, just replace the actual function with the test and be done with it.
Re:The tragedy of 100% code coverage (2016) by jameshart on Tuesday May 09, 2017 @08:35AM (#14299438)
One answer to that is to apply property based testing. If you're testing an addition algorithm, for example, you might test that

   A + B == B + A
   A + 0 == A
   (A + B) + C == A + (B + C)
   A + B != A + C (given B != C)
Re:The tragedy of 100% code coverage (2016) by slavik81 on Tuesday May 09, 2017 @11:55AM (#14301080)
It's a good idea in principle, though your third and fourth properties are not true of floating point addition.

When you have one available, testing against the results against a reference implementation is a little more direct. Though, given how easy those property tests are, you might as well do both.

Re:The tragedy of 100% code coverage (2016) by detaro on Tuesday May 09, 2017 @07:26AM (#14299066)
Your comparison source could be running on a different platform, or be slower, or ... and thus not be a drop-in replacement. (E.g. in the example of a broken floating-point unit, compare the results with a software emulation)
Re:The tragedy of 100% code coverage (2016) by StavrosK on Tuesday May 09, 2017 @08:09AM (#14299294)
Hmm, true, it may have been a different platform/slower, thanks. It can't have been hardware/software, as the GP said they were testing a library (which implies software).
Re:The tragedy of 100% code coverage (2016) by dagw on Tuesday May 09, 2017 @10:54AM (#14300523)
How is the algorithm supposed to know what the right thing to return is

Some times I have, for example, matlab code that I 'know' is right and want to verify that my C/Python/Julia... code does the same thing. So then I just call the matlab code, call the new code and check if they're the same.

Or I have a slow exact method and want to see if my faster approximation always gives an answer 'close enough' to the correct answer.

Re:The tragedy of 100% code coverage (2016) by stephencanon on Tuesday May 09, 2017 @11:52AM (#14301050)
With floating-point math libraries especially, it's frequently the case that it's relatively easy to compute the desired result accurately by using higher precision for internal computations; e.g. you might compare against the same function evaluated in double and then rounded to float.

This sounds pretty sketchy at first, but it turns out that almost all of the difficulty (and hence almost all of the bugs) are in getting the last few bits right, and the last few bits of a much-higher precision result don't matter when you're testing a lower-precision implementation, so combined with some basic sanity checks this is actually a very reasonable testing strategy (and widely used in industry).

Re:The tragedy of 100% code coverage (2016) by praptak on Tuesday May 09, 2017 @07:20AM (#14299045)
If a function takes a single 32 bit argument you can run it on every possible value.
Re:The tragedy of 100% code coverage (2016) by detaro on Tuesday May 09, 2017 @07:23AM (#14299054)
… this reply does nothing to answer the parent's question. It's obvious that you can run it on all possible values, but that doesn't answer the question of the return values. (unless you only want to verify "doesn't crash" or something similarly basic)
Re:The tragedy of 100% code coverage (2016) by praptak on Tuesday May 09, 2017 @10:37AM (#14300355)
I believe the question was edited after I wrote my answer. This, or I just misread it - unfortunately HN doesn't let us know. Anyway, I'm providing another answer to the question as it is now.
Re:The tragedy of 100% code coverage (2016) by praptak on Tuesday May 09, 2017 @10:45AM (#14300436)
Validating the answer is different than providing it:

   FLOAT_EQ(sqrt(x) * sqrt(x), x) && sqrt(x)>=0.0
is a pretty good test for the square root even though it doesn't compute it.