Re:Text to Image Synthesis Using Thought Vectors
by Y_Y on Friday August 26, 2016 @11:44AM
(#12367003)
I can see this being useful for police sketch artists.
[ Parent
| on HN ]
Re:Text to Image Synthesis Using Thought Vectors
by paarthn on Friday August 26, 2016 @12:04PM
(#12367146)
There is still a long way to go, to be able to do that. The model currently generates 64 X 64 pictures and is trained on a very specific flowers image dataset. Nevertheless, it would be a great idea to experiment with such a dataset (of sketches and descriptions) if available.
Re:Text to Image Synthesis Using Thought Vectors
by taneq on Friday August 26, 2016 @11:05PM
(#12370735)
But the thing about machine learning is that once it works at all, "a long way to go" generally means "add more training data" rather than "we require significant conceptual breakthroughs".
Re:Text to Image Synthesis Using Thought Vectors
by jcannell on Saturday August 27, 2016 @02:49PM
(#12373612)
That is not generally true. For example, GANs work great on MNIST, pretty well on the flower dataset, and ok on bedrooms.
But the same techniques currently fail on ImageNet - which actually is a much larger dataset. "Add more training data" is not a magic solution that overcomes limitations of your model.
In particular if you look at the generative models these GANs map to, it makes sense that they can learn 2D shapes and texture patterns, but rendering a complex 3D scene with significant depth complexity and lighting interactions is an entirely different beast. That problem has been studied deeply in 3D computer graphics and the generative programs successful there are vastly more complex than current GANs.
Re:Text to Image Synthesis Using Thought Vectors
by hughperkins on Saturday August 27, 2016 @06:30PM
(#12374447)
seen this? https://arxiv.org/abs/1605.09304 awesome generated images, from imagenet
Re:Text to Image Synthesis Using Thought Vectors
by gwern on Friday August 26, 2016 @12:20PM
(#12367262)
A GAN would be helpful more for its latent space abilities than for any text-to-image capability. If you look at the GAN papers, you can see that they demonstrate how you can warp an image across multiple semantic dimensions like adding/remove eyeglasses, darkening hair, turning frowns into smiles etc. This would be great for sketches because you can have the victim walk through the latent space by recognizing which version looks more like the attacker, since people can recognize faces far better than they can verbalize detailed descriptions.