If you have an oracle, you don't need to know the net specs.

An oracle, here, would be any version of the system that you can query against repeatedly without suffering too much of a penalty.

That's a good point! But your proposed solution is security by obscurity. What if a rogue employee trained a backdoor into your DNN? What if the NSA asked vendors for a master key?

The purpose of this research is to work as a proof of concept. Sure, this iteration needs to cheat slightly to achieve its results. However, looking at the images that aren't just noise, it seems to be possible to construct less specialized images that fool many nets.

The paper does not use the weights of the network to produce (nearly all of) these images (see the responses to Karpathy above).

> In theory, you could do the same for people's brains.

No because the instant you used an image that was close, but wrong the human brain would retrain.

I wonder in the neural net can do the same - try this experiment on an active neural net and let it train itself. (i.e. don't tell it it's being faked, let it figure it out then correct for it).

No, the images generated worked on even different neural networks trained on different data.