prajit

> Or any co/startup doing "deep tech" in the area of Deep Learning.

Send me a message, my email is in my profile.

If you're interested in Sequence to Sequence tasks (e.g. neural machine translation or abstractive summarization) with small data, check out our recent paper from Google Brain tackling this problem (disclaimer, I'm the first author): https://arxiv.org/abs/1611.02683

Yes, there are two types of parallelism: model and data. Data parallelism is simply training the the model on multiple computers with different minibatches, and aggregating the gradients. Model parallelism is hosting different parts of the model on different computers. Of course, these two parallelisms can be combined. A good explanation is the One Weird Trick paper: http://arxiv.org/abs/1404.5997

I implemented this system for the final project of my computer vision class. Couldn't get it to work by the deadline, but I'm very familiar with this paper. Happy to answer any questions.

"There’s the famous observation that the value of a network grows as a function of the square of the number of nodes"

The value of a network is measured as O(V^2), where V is the number of nodes (users) in the network.

"and also many of these services/products double their userbase every N months."

V = 2^N (the number of users doubles every N months), so the overall function for the value of a network is O(V^2) = O((2^N)^2) = O(2^(2N)) = O(4^n). O(4^n) > O(2^n), but it is not the same as O(2^(N^2)), which was what Sam was trying to convey.

A question about the actual slides: why don't they use unsupervised pretraining (i.e. Sparse Autoencoder) for predicting MNIST? Is it just to show that they don't need pretraining to achieve good results or is there something deeper?

This is great. Thank you.

This is not "first come, first serve", right?

Type in the Konami Code and see what happens (up, up, down, down, left, right, left, right, b, a) - an Easter Egg!