throwaway_bob

FAIR, DeepMind, Brain, MSR and many others all employ large numbers of people working on pure research with no immediate business application, and with at least as much freedom as their academic counterparts.

honestly, then y'all are mostly responsible. If you are going to have a younger student do a guest blog post, you should make sure that the project makes sense technically, or at least is not silly. What if the student tried to train a neural net to build a perpetual motion machine and reported their progress on that? Would you have posted it? That project would be about the same level of silliness as what is reported here. I don't blame them at all for doing a silly thing, but I surely blame you for making it your guest blog post.

I think you are missing the parent's point. A CNN is a more complicated model, not a simpler one- it is better to try simple linear classifiers on bags of word or bags of bigrams or trigrams before breaking out the more complicated neural models. Note that you can do this easily with VW or FastText.

The exact phrasing matters too, I guess. There are lots of ways to find a good packing, and even good bounds on how far your good packing is from the global optimizer. So a reasonable question to your interviewer would have been "does it need to be the exact global optimizer, or can it be a good approximation ?". If they wanted a good approximation it might not be a bad question, and the question may have been designed to get you to ask for the distinction? Or maybe they were just dicks, who knows.

Note that the fact that this can be easily accomplished in VW doesn't really take away from the message the authors are trying to make; namely, simple models done carefully are nearly as (or more) effective in these sorts of problems as fancy deep models, but much cheaper to train and test.

I lived on henry street for a while, and I can't say that I know anything about the back end of the distribution network, but I can say this article is missing a lot of things about the front end. The produce is cheaper than a standard western store, but the quality is much more varied. Moreover, the market (in the economic sense) seems really efficient. If you see something cheap in Manhattan chinatown in every store, it might be a really good deal, and just that thing is in season or a lot of it was just delivered. But if something is cheap in just one store, it is probably in bad shape. Stores are willing to sell produce that simply wouldn't be sold in ordinary western stores; and do things I think wouldn't be allowed elsewhere. For example, I have seen one place where if they have baskets of strawberries, and some of the strawberries are getting moldy, they will by hand separate out the non moldy ones, toss the moldy ones, and repackage the baskets. Moreover, the stores don't bother to keep clean at all, you can smell them from a block away. If you like food shopping, and you pay attention, its great, but if you want to just get your food, it is a lot of work. I think most americans do not want this kind of tradeoff.

You do realize that these things were already being studied in the 90's, right? The Grefenstette paper mentioned in this article makes use of an architecture from Das et al. in 1992. More generally, people have been talking about functional programming and relationships with AI since there was functional programming. However nice Colah's blog post is, he most certainly did not come up with any of these ideas first.

when he says "probability", what space does he mean? This analysis does not seem very careful.

any visualization of these algorithms in 2 dimensions (with cubic feature expansion!) is completely misleading if you intend to work on any real problem with many dimensions. Also, for those asking for execution times, these would be horribly misleading as well.

Just because wired says something doesn't make it true. Whatever other accomplishments he may have, Le was not a contributor to word2vec, and he did not originate the idea of mapping words to vectors.

pacman already takes place on one of these tori

the fact that random noise can be classified as strongly belonging to some class, and the fact that classification results can be unstable, is simply a result of the fact the input space is very high dimensional, and the output space is very small (say a few isolated points). That is, if you are discriminatively training a mapping from images in R^(224 x 224 x 3) to 1000 points (class labels), there is going to be a tremendous amount of instability in the inverse direction.