News for hackers. Stuff that matters.
Re:Why CVE-2022-3602 was not detected by fuzz testing by jeffbee on Monday November 21, 2022 @11:22AM (#33694294)
It is trivial to enforce that new functions have new unit tests and fuzz tests. You are the reviewer of https://github.com/openssl/openssl/pull/9654 and you just say "Please add unit tests and fuzz tests for foo and bar" and you don't approve it.

I don't know what the deal is with their testing culture but in year 27 of the project they demonstrably haven't learned this lesson. It's nice that they added integration tests (testing given encoded certs) but as the article points out that was insufficient.

Re:Why CVE-2022-3602 was not detected by fuzz testing by dralley on Monday November 21, 2022 @11:44AM (#33694622)
IMO, one of the biggest benefits of "modern" systems languages like Rust, D, Zig is how much easier they make writing and running tests compared to C and C++. Yes, you can write tests for those languages, but it's nowhere near as trivial. And that makes a difference.
Re:Why CVE-2022-3602 was not detected by fuzz testing by pjmlp on Monday November 21, 2022 @12:31PM (#33695284)
I was writing unit tests for C in 1996, naturally we still haven't coined the term, so we just called them automatic tests.

It was part of our data structures and algorithms project, failure to execute the automatic tests meant no admission to the final exam.

We had three sets of tests, those provided initially at the begin of the term, those that we were expected to write ourselves, and a surprise set on the integration week at the end of the semester.

Re:Why CVE-2022-3602 was not detected by fuzz testing by fisf on Monday November 21, 2022 @12:07PM (#33694945)
I am not buying it.

Writing unit tests for c/c++ is trivial. There are perfectly fine test frameworks, used by developers every day, integrated in any major IDE or runnable as one-liner from the command line.

This is absolutely a cultural problem.

Re:Why CVE-2022-3602 was not detected by fuzz testing by edgyquant on Tuesday November 22, 2022 @12:43AM (#33702454)
In modern languages you either have built in testing or it’s a single package install away. It is nowhere near this simple with C or C++
Re:Why CVE-2022-3602 was not detected by fuzz testing by bluGill on Monday November 21, 2022 @05:03PM (#33698682)
Last week I code a "please add unit tests for this code". The person who wrote that comment wasn't aware that this was a refactoring where the functionality was well tested.

There is no substitute for reviewers who really understand the code in question. The problem is they are the ones writing the code and so are biased and not able to give a good review.