Images that fool computer vision raise security concerns

Re:Images that fool computer vision raise security concerns by wodenokoto on Tuesday March 24, 2015 @12:05AM (#9254839)

I've quite enjoyed reading your paper since it was uploaded to arxiv in December and I have been toying with redoing the MNIST part of your experiment on various classifiers. (I'm particularly interested to see if images generated against an SVM can fool a nearest neighbor or something like that.)

But I'm having problems generating images: A top SVM classifier on MNIST has a very stable confidence distribution against noisy images. If I generate 1000 random images, only 1 or 2 of them will have confidences that are different from the median confidence distribution. That is, all the images are classified with the same confidence as class 1. They also share the same confidence for class 2, etc.

So it is very difficult to make changes that affect the output of the classifier.

Any tips on how to get started with generating the images?

[ Parent | on HN ]

Re:Images that fool computer vision raise security concerns by jeffclune on Tuesday March 24, 2015 @03:16PM (#9258779)

I would just unleash evolution. 1 or 2 in the first generation is a toehold, and from there evolution can begin to do its work. You can also try a larger population (e.g. 2000) and let it run for a while.

[ Parent | on HN ]