News for hackers. Stuff that matters.
Re:Bugs found by jsfunfuzz by wldlyinaccurate on Wednesday February 25, 2015 @05:31AM (#9106130)
This is really interesting, and a great approach to testing. I wonder if other JS engines like V8 are using this or something similar?
Re:Bugs found by jsfunfuzz by amelius on Wednesday February 25, 2015 @08:05AM (#9106509)
Indeed interesting. I wonder if there is a tool for generating input in a way that guarantees that all locations in the program are actually covered (in other words, that all reachable code has been reached).

Of course, this is no guarantee that the program actually works, but it would make me sleep better :)

Re:Bugs found by jsfunfuzz by SloopJon on Wednesday February 25, 2015 @08:38AM (#9106627)
An instrumented fuzzer like AFL purports to do this:

http://lcamtuf.coredump.cx/afl/

This blog post is a fascinating description of its potential:

http://lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-thi...

I haven't had as much success with it, but it's so interesting that I'll keep trying. I'm also interested in KLEE, which I found in a similar HN story, but it has very specific build requirements:

https://klee.github.io

[Edit: bhouston posted the exact same links minutes before I did. Anyway, cool stuff.]

Re:Bugs found by jsfunfuzz by ainsej on Wednesday February 25, 2015 @08:52AM (#9106695)
KLEE is pretty cool. To compensate for the atrocious build instructions there's a docker image which contains KLEE built and ready to use (https://registry.hub.docker.com/u/kleeweb/klee/).

There's also a web interface to just play around with KLEE without having to download and install anything that a few other people and I worked on available at http://klee.doc.ic.ac.uk:55080/, which we open-sourced https://github.com/klee-web/klee-web.

Re:Bugs found by jsfunfuzz by falcolas on Wednesday February 25, 2015 @08:40AM (#9106637)
One type of testing (sorry, not a tool yet) which can help with this is combinatorial testing. Basically, when testing against a set of multiple inputs, create a test which will test every possible combination of input for a pair of inputs.

i.e. for inputs A, B, C, try every combination of A-B, A-C, and B-C

It sounds like a lot, but its not too onerous with the speed of computers we have today. This type of testing will flush out most bugs which will result from a particular combination of inputs, and is deterministic, when compared to fuzz testing.

This is not to detract from fuzz testing, just a note that it is effectively designed to be non-deterministic, which can result in a delay in bug detection.

Re:Bugs found by jsfunfuzz by mannykannot on Thursday February 26, 2015 @02:47PM (#9114928)
Just to be clear, taking inputs in pairs does not guarantee complete coverage, even when trying all permutations of those pairs. I mention this because it is what the OP of this thread asked about.
Re:Bugs found by jsfunfuzz by bhouston on Wednesday February 25, 2015 @08:34AM (#9106606)
There is a tool like that for compiled binaries. Details on its usage here:

http://lcamtuf.coredump.cx/afl/

http://lcamtuf.blogspot.ca/2014/11/pulling-jpegs-out-of-thin...

Re:Bugs found by jsfunfuzz by spork1 on Wednesday February 25, 2015 @07:23AM (#9106376)
Yes, they are running various internal testing suites in google, check here for example: https://code.google.com/p/chromium/issues/detail?id=360298