Re:Where will the next major advance towards general purpose...
by nicklo on Thursday September 22, 2016 @11:05AM
(#12557285)
Deep image captioning is essentially what you are describing.
Check out:
Re:Where will the next major advance towards general purpose...
by Matumio on Thursday September 22, 2016 @11:29AM
(#12557528)
Try looking for papers about image caption generation. For machine translation you can squeeze the information of a sentence into a vector (using a LSTM). You can then reconstruct the sentence in a different language. There was a paper how to generate such a "sentence representation vector" from an image instead, translating from "image" to "english".
Re:Where will the next major advance towards general purpose...
by dumitrue on Thursday September 22, 2016 @11:13AM
(#12557350)
Apart from the captioning work already mentioned, there's also visual analogy making work by Reed et al. http://www-personal.umich.edu/~reedscot/nips2015.pdf which is pretty exciting (sort of a converse to what you're proposing).
In fact the word associations could give it clues about what's in the image. For example it sees a picture of woman standing next to a king. So it uses word2vec to guess King + woman = queen.
What do you guys think? Anyone want to build a little prototype?