The problem is that the adversarial examples they are able to come up with are very close to the original images, so this means the smoothness assumption seem to be invalid for deep learning models. As they put it in the paper : "Our main result is that for deep neural networks, the smoothness assumption that underlies many kernel methods does not hold."
It's going to be interesting to see what happen when other researchers try to replicate the results for other models and datasets.
To accomplish that, for example, they could take perturbations of the input and then e.g. take a majority majority vote.
However, maybe there's a middle ground. I.e. maybe we don't need the more esoteric features that SIFT uses, but it just makes sense to do edge detection, and say a Fourier transform for audio.
Normalization and other data transforms are still required for discovery of features.
The impl is a convolutional RBM. That being said, where's your deep learning library? ;)
The documentation is still being worked on. If that's the only thing you can cherry pick from a new project, I must not be doing too bad.
Edit: Clarified. I know you're just a troll account, but I'll throw this out there anyways, I think anyone who judges the quality of a lib based on a 2 second reading isn't qualified to judge much. A lot of it does rely on me with the documentation, but I'd love to chat with you one on one if you think I'm not qualified. Trash me all day, I'll either learn something or embarrass you. Both are fine with me ;).
I would probably recommend Torch at this point. The incentives don't exist for the experts to make really good open source projects and spend all the time required maintaining them and helping people with them.
You're right about this which is why I started a company around it.
I've already talked with andrew ng and yoshua bengio. My incentives are different from there's, however, I do have their blessings to continue doing this.
I walked in to this expecting skeptics. That being said, I love deep learning as a field and will be implementing every possible neural net I can. Since my incentives are different, I can explore the different use cases with customers and help further the field in different directions that might not make sense for say baidu, facebook ,or google.
More specifically, adding SIFT or edge detection to your raw pixel input will almost always strictly improve a deep model's performance (though they might be redundant) at a not-particularly-large computational cost.
It wouldn't solve the adversarial example problem though, except to the extent that it makes calculating gradients harder.
Yes and just to clarify - this seems like an indication that however useful, deep learning can't follow through with that promise. And I think the ability to do this the key thing - all the approaches work at some level but without this "automagicity", each become hostage to brow-wrinkling experts who become the only one to understand the black-magic of algorithm tuning.
Any algorithms have weaknesses. It's a matter of being aware of them and compensating for them in your model, possibly by using another model.