Re:A New Twist on Neural Networks
by dr_zoidberg on Thursday November 02, 2017 @12:07PM
(#15611624)
No, the problem I was pointing at is that you want to change a part of the image (horse into zebra), but the style transfer GAN learned to map pixels from one space into another. So it knows it has to change the colour of some things, and add stripes here and there, and also probably there too. But it isn't consistent with the stripes (you can see in some gifs how the pattern suddenly changes and adjusts), and it doesn't recognize (segment) the horse as the only relevant thing that has to be changed.
But that's not bad per se about style transfer. It's an interesting technique, but if you want to convert all horses to zebras in an image, that seems to be a bit too general for current-generation GAN architectures. Maybe it can be improved upon, or a different, novel architecture is required, and not just something you can solve by throwing more data at it.
+ Brown horses on beaches
+ Some way to indicate "white" "horse".
If you have a good idea about how we can train for your problem, I'm not so convinced that it cannot be solved.