Re:Stranger Strings: An exploitable flaw in SQLite
by phoe-krk on Tuesday October 25, 2022 @11:35AM
(#33332006)
> Your test suite may not be able to hit it either.
Minor nitpick, process terminations are unit-testable in general; see e.g. https://github.com/google/googletest/blob/main/docs/advanced...
Re:Stranger Strings: An exploitable flaw in SQLite
by garaetjjte on Tuesday October 25, 2022 @12:26PM
(#33332749)
They don't count assertions in testing coverage: https://www.sqlite.org/testing.html#coverage_testing_of_defe...
Re:Stranger Strings: An exploitable flaw in SQLite
by jonhohle on Tuesday October 25, 2022 @12:32PM
(#33332840)
What is fair in testing and what is being tested? If I can stub `secure_hash` and `bar`, that’s a pretty easy test to write. Assuming the symbols are visible at test time, stubbing those is relatively straight forward in C, Java, Bash, and Ruby, JavaScript, etc.
Even in the above examples, a linter should point out that the second `p == null` is inaccessible. I’m very pro test coverage, but static analysis shouldn’t be thrown out just because unit tests are available.
I don’t think you can do that with every project. For example, a function f may do:
If all callers also do that test, there’s no way to hit that abort() line in the program. Your test suite may not be able to hit it either. Maybe f is a private function, or that test occurs on one of its intermediate values, or f accidentally has two identical tests: Even if it is possible to hit that line, finding a path through the code that passes in a null value can be extremely hard.As a simple contrived counterexample, try getting 100% coverage on a function that does if(secure_hash(foo) == 0) bar();