News for hackers. Stuff that matters.
Re:Stranger Strings: An exploitable flaw in SQLite by Someone on Tuesday October 25, 2022 @11:08AM (#33331594)
> Give me one project and I'll send you a test suite that "tests" 100% of executable lines.

I don’t think you can do that with every project. For example, a function f may do:

  if (p == null) {
    abort();
  }
If all callers also do that test, there’s no way to hit that abort() line in the program. Your test suite may not be able to hit it either. Maybe f is a private function, or that test occurs on one of its intermediate values, or f accidentally has two identical tests:

  if (p == null) {
    abort(“p is null”);
  }
  if (p == null) { // copy-paste bug
    abort(“q is null”);
  }
  if (r == null) {
    abort(“r is null”);
  }
Even if it is possible to hit that line, finding a path through the code that passes in a null value can be extremely hard.

As a simple contrived counterexample, try getting 100% coverage on a function that does if(secure_hash(foo) == 0) bar();

Re:Stranger Strings: An exploitable flaw in SQLite by phoe-krk on Tuesday October 25, 2022 @11:35AM (#33332006)
> Your test suite may not be able to hit it either.

Minor nitpick, process terminations are unit-testable in general; see e.g. https://github.com/google/googletest/blob/main/docs/advanced...

Re:Stranger Strings: An exploitable flaw in SQLite by garaetjjte on Tuesday October 25, 2022 @12:26PM (#33332749)
They don't count assertions in testing coverage: https://www.sqlite.org/testing.html#coverage_testing_of_defe...
Re:Stranger Strings: An exploitable flaw in SQLite by jonhohle on Tuesday October 25, 2022 @12:32PM (#33332840)
What is fair in testing and what is being tested? If I can stub `secure_hash` and `bar`, that’s a pretty easy test to write. Assuming the symbols are visible at test time, stubbing those is relatively straight forward in C, Java, Bash, and Ruby, JavaScript, etc.

Even in the above examples, a linter should point out that the second `p == null` is inaccessible. I’m very pro test coverage, but static analysis shouldn’t be thrown out just because unit tests are available.