jeffclune

Hello all. We hope you enjoy the paper.

Here are two fun gifs of clever solutions AI came up with:

https://twitter.com/jeffclune/status/974718199722795008

https://twitter.com/jeffclune/status/973605950266331138

Here are some press articles which provide a shorter summary of some fun anecdotes.

popularmechanics.com/technology/robots/a19445627/the-hilarious-and-terrifying-ways-algorithms-have-outsmarted-their-creators/

https://www.newscientist.com/article/8-8-hilarious-ways-ai-h... (paywalled unfortunately)

Hello all. I am one of the paper's coauthors. Thank you for your interest in this work! We hope you enjoy it. Just an FYI, this paper is part of a cluster of five papers released on Tuesday morning, detailed in this blog post: https://eng.uber.com/deep-neuroevolution

No, it does not. See my reply to similar comments.

I would just unleash evolution. 1 or 2 in the first generation is a toehold, and from there evolution can begin to do its work. You can also try a larger population (e.g. 2000) and let it run for a while.

> Likewise, I was not very surprised that you can produce fooling images, but it is surprising and concerning that they generalize across models. It seems that there are entire, huge fooling subspaces of the input space, not just fooling images as points. And that these subspaces overlap a lot from one net to another,

Agreed. That is surprising, and also increases the security risks, because I can produce images on my in-house network and then take them out into the world to fool other networks without even having access to the outputs of those networks.

> likely since they share similar training data (?) unclear.

The original Szegedy et al. paper shows that these sort of examples generalize even to networks trained on different subsets of the data (and with different architectures).

> Anyway, really cool work :)

Thanks. :-)

See above. These images were accepted to a juried art competition and posted in a museum:

http://www.evolvingai.org/share/20150213_184537.jpg

Here they are: http://www.evolvingai.org/fooling

There is also a video summary of the paper there.

The paper does not use the weights of the network to produce (nearly all of) these images (see the responses to Karpathy above).

In fact, we submitted them to an art competition and they were accepted and put up in a museum!

http://www.evolvingai.org/share/20150213_184537.jpg

Also, you don't necessarily need a lot of predictions from a net to fool it, because (another surprising result) images that fool one net tend to fool others! So I can create fooling images on my in-house net and then take them and fool your net-used-for-some-important-application without getting any feedback from your net. That's very surprising, and does raise serious security concerns.

You say that these images are highly optimized to produce this effect and would not occur by chance, but have you looked at the images in the "fooling" paper?

http://www.evolvingai.org/fooling

Some of them are very simple, and DO occur a lot in the world. For example, the alternating yellow and black line pattern would be encountered by a driverless car, and it would think it is seeing a school bus.

Surprisingly, ensembles do not really help. We have tried this and it does not work (the final paper for CVPR 2015 will show these results). Also see the work of Szegedy et al. and Goodfellow et all, which also show that ensembles do not really help.

Ha. Jason and I were typing at the same time. :-)

They would look like optical illusions!

Hi Andrej,

I agree with most of what you say, but note that nearly all of the images in the paper were generated without the gradient. I.e. all the images produced by evolution did not use the gradient, only the output of the network regarding its prediction confidence. There are some images that use the gradient, but only to show a 3rd class of "fooling images".

PS. It's nice to see our work (both this paper and the NIPS paper on transfer learning) in your class. Thanks for including it. I wish I could have my students take your course!

Sorry. The site is getting hammered right now due to the press articles and Hacker News. We just did a bit of hacking and hopefully it will work now!

No. People just tend to choose ones that are.

Here are some that are mostly bilaterally symmetrical, but with left-right asymmetry.

http://endlessforms.com/o/69562/ (ghost) http://endlessforms.com/o/60967/ (mad hatter) http://endlessforms.com/o/39225/ (Dr. Evil)

There are some with no symmetry at all....

Here is a longer article, with pictures of evolved objects printed in Silver and Bronze:

http://www.physorg.com/news/2011-08-endless-web-d-printable....

That's because you know how to use Blender, which most people don't. But the site is not for a situation when you have a specific design in mind, instead it is for when you want to explore the space of possible designs and see suggested designs that you might not think of on your own, such as this: http://endlessforms.com/o/372201/

But I also doubt I could produce all of these shapes in Blender. I would have no idea how to do these, for example:

http://endlessforms.com/o/103181/ http://endlessforms.com/o/82117/

Best, Jeff

I noticed a surge in traffic to EndlessForms.com and realized it was coming from this post. Thanks to Joel for mentioning our site[1]! For those of you that don't know what I am talking about, see Joel's comment on the original post where he recommends EndlessForms.com, where you can design objects with evolution and 3D print them. Thanks all for checking it out.

[1] http://endlessforms.com

You can control the rotation with your arrow keys. Does that accomplish what you want? We need to make that more obvious. Thanks for the feedback!