Images that fool computer vision raise security concerns

Re:Images that fool computer vision raise security concerns by yosinski on Monday March 23, 2015 @04:01PM (#9253036)

No, but we did make use of (1) a large number of input -> network -> output iterations, along with (2) precisely measured output values to decide which input to try next. It may not be so easy to experiment in the same way on natural organisms (ethically or otherwise).

Of course, if you're as clever as Tinbergen, you might be able to come up with patterns that fool organisms even without (1) or (2):

https://imgur.com/a/ibMUn

[ Parent | on HN ]

Re:Images that fool computer vision raise security concerns by JoeAltmaier on Monday March 23, 2015 @04:09PM (#9253081)

Perhaps a single experiment on millions of different people? A web experiment of some kind? "Which image looks more like a panda?" and flash two images on the screen.

[ Parent | on HN ]

Re:Images that fool computer vision raise security concerns by yosinski on Monday March 23, 2015 @05:03PM (#9253345)

That's a good idea, though note that there's a difference between asking "Which of these two images looks more like a panda?" and "Which of these two images looks more like a panda than a dog or cat?". The latter is the supervised learning setting used in the paper, and generally could lead to examples that look very different than pandas, as long as they look slightly more like pandas than dogs or cats. The former method is more like unsupervised density learning and could more plausibly produce increasingly panda-esque images over time.

A sort of related idea was explored with this site, where millions (ok, thousands) of users evolve shapes that look like whatever they want, but likely with a strong bias toward shapes recognizable to humans. Over time, many common motifs arise:

http://endlessforms.com/

[ Parent | on HN ]

Re:Images that fool computer vision raise security concerns by tripzilch on Wednesday March 25, 2015 @07:11PM (#9266699)

Problem is, even if you succeed and end up with a fabricated picture that fools human neural nets into believing it's a picture of a panda, how would you tell it's not really a picture of a panda?

You'd need another classifier to tell you "nope it's actually just random noise and shapes" ... hm.

[ Parent | on HN ]

Re:Images that fool computer vision raise security concerns by JoeAltmaier on Tuesday March 31, 2015 @04:26PM (#9299652)

Probably not hard - just engage the higher cognitive functions. "Does it look like noise? Yeah." Or just close one eye and look again.

[ Parent | on HN ]

Re:Images that fool computer vision raise security concerns by tripzilch on Tuesday April 07, 2015 @08:03AM (#9333317)

I think you missed my somewhat deeper philosophical point :)

Who gets to decide what is really a picture of a panda?

If we'd manage to craft a picture that could with very high certainty trick human neural nets (for the sake of argument, including those higher cognitive functions) into believing something is a picture of a panda, "except it actually really isn't", what does that even mean?

Human insists it's a picture of a panda, computer classifier maintains it's noise and shapes.

Who is right? :)

[ Parent | on HN ]

Re:Images that fool computer vision raise security concerns by JoeAltmaier on Sunday April 12, 2015 @12:02AM (#9362104)

Interesting, sure. But I started out wondering if some obviously-noise picture could be found that fooled humans, at least at first glance. "Hey a panda! Wait, what was I thinking, that's just noise!" It would be weird and cool, on the order of the dress meme etc. but much more so.

Kind of like the memes in Snowcrash, ancient forgotten symbols that make up the kernel of human thought.

[ Parent | on HN ]

Re:Images that fool computer vision raise security concerns by irascible on Tuesday March 24, 2015 @08:40AM (#9256178)

Like a picture of a blue and black dress?

[ Parent | on HN ]