News for hackers. Stuff that matters.
Re:Afl-fuzz: making up grammar with a dictionary in hand by darkmighty on Wednesday April 15, 2015 @08:05AM (#9380337)
Wouldn't overfitting be a huge issue? You'd probably have to put a large penalty on size/running time vs correctness.

Getting trivial solutions is really easy and doesn't "connect" much to the "space" of good algorithms. That's the primary reason for the development of neural networks: with continuous differentiable elements you can do a nicer "hill climbing" toward small representations through back-propagation. With hard-value logic you'd be more stumbling in the dark until you get lucky, but that's O(exp(N)) on the program size.

Re:Afl-fuzz: making up grammar with a dictionary in hand by logicallee on Wednesday April 15, 2015 @08:22AM (#9380429)
this is a very good point. I suppose you could in some cases make the test cases "exhaustive" (or at least drawing from all possible inputs) which would possibly mitigate this problem. Putting a limit on size/running time as you suggest is certainly a good idea, though it's not like any algorithm can brute-force more than a couple of composed tokens at a time....
Re:Afl-fuzz: making up grammar with a dictionary in hand by TheLoneWolfling on Wednesday April 15, 2015 @08:14AM (#9380387)
Just write a test case generator!

(Now you have two problems)

Re:Afl-fuzz: making up grammar with a dictionary in hand by darkmighty on Wednesday April 15, 2015 @08:17AM (#9380410)
Yes that's a solution, but it might be called an optimizer then :)