Re:Rethinking software testing: perspectives from the world ...
by triska on Sunday July 11, 2021 @05:47AM
(#27799596)
To clarify: Tests as I see them and described them above do not need to stop when the software is shipped. They are meant to run indefinitely and can continue also long after the software is shipped, and independent of the environment in which it was developed. For example, I found several mistakes in released versions of GNU Emacs with test cases that repeatedly do the same thing, over and over, until they reveal a crash or memory leak. A few examples of this approach:
Fuzzing is a useful approach and does reveal many issues with software. As I see it, the fact that fuzzing already tends to reveal mistakes in software is a good motivation to strive for methods that yield greater coverage based on systematic exhaustive search, so that also mistakes that occur in very rare situations are reliably detected.
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=496
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=576
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=726
Fuzzing is a useful approach and does reveal many issues with software. As I see it, the fact that fuzzing already tends to reveal mistakes in software is a good motivation to strive for methods that yield greater coverage based on systematic exhaustive search, so that also mistakes that occur in very rare situations are reliably detected.