Squashed Bugs, with Failure Rate Heatmaps

Re:Squashed Bugs, with Failure Rate Heatmaps by pfdietz on Friday December 20, 2019 @11:57PM (#21849713)

One thing I've wondered about with this sort of language fuzzing is how well the fuzz inputs cover the program under test. They had all the tools in place to measure that, but I didn't see them report how well they did.

Ideally, one would like to adjust the fuzzer based on that information to steer executions to lesser covered parts of the code (or, possibly, toward parts you want to focus your testing on.)

[ Parent | on HN ]

Re:Squashed Bugs, with Failure Rate Heatmaps by iamxy on Saturday December 21, 2019 @01:50AM (#21850022)

Author here. Thanks for your great advice. We will apply this experimental project to testing TiDB in the future and output the report to https://github.com/fuzzdebugplatform/fuzz_debug_platform/iss.... Through the statistics of code block coverage, we can not only identify suspicious code blocks but also count code coverage.

The fuzzer we implemented is driven by BNF expressions. We can adjust the inputs of the fuzzer based on the statistics.

[ Parent | on HN ]

Re:Squashed Bugs, with Failure Rate Heatmaps by pfdietz on Saturday December 21, 2019 @08:14AM (#21851077)

One of the tricks you should try is adding Swarm Testing. It tends to make this sort of fuzzing more effective at finding bugs, and I would not be surprised if it improves your coverage also.

https://dl.acm.org/citation.cfm?id=2336763

https://blog.regehr.org/archives/591

[ Parent | on HN ]

Re:Squashed Bugs, with Failure Rate Heatmaps by iamxy on Saturday December 21, 2019 @09:31AM (#21851312)

Thank you for the information, very helpful, I will learn and try it in our test.

[ Parent | on HN ]