The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies

Re:The Sketchy Database: Learning to Retrieve Badly Drawn Bu... by ljf on Saturday October 01, 2016 @07:00PM (#12620279)

Great to site but the title is all wrong, these aren't photo realistic images from drawings, this is taking a sketch, classifying it and trying to find a similar photo.

From the site

Abstract We present the Sketchy database, the first large-scale collection of sketch-photo pairs. We ask crowd workers to sketch particular photographic objects sampled from 125 categories and acquire 75,471 sketches of 12,500 objects. The Sketchy database gives us fine-grained associations between particular photos and sketches, and we use this to train cross-domain convolutional networks which embed sketches and photographs in a common feature space. We use our database as a benchmark for fine-grained retrieval and show that our learned representation significantly outperforms both hand-crafted features as well as deep features trained for sketch or photo classification. Beyond image retrieval, we believe the Sketchy database opens up new opportunities for sketch and image understanding and synthesis.

[ Parent | on HN ]

Re:The Sketchy Database: Learning to Retrieve Badly Drawn Bu... by ljf on Saturday October 01, 2016 @07:01PM (#12620283)

In fact the original title of the page is: The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies Patsorn Sangkloy Nathan Burnell Cusuh Ham James Hays

[ Parent | on HN ]

Re:The Sketchy Database: Learning to Retrieve Badly Drawn Bu... by DanielleMolloy on Saturday October 01, 2016 @09:24PM (#12620660)

This is a recent paper that actually creates photos from sketches of faces, using deep nets: https://arxiv.org/abs/1606.03073

Figure 4 does this for self portraits of some famous artists.

[ Parent | on HN ]

Re:The Sketchy Database: Learning to Retrieve Badly Drawn Bu... by aab0 on Saturday October 01, 2016 @09:46PM (#12620723)

You could probably apply the same code. The dataset ("acquire 75,471 sketches of 12,500 objects") sounds adequate, and if not, can be boosted by first training a CNN to do photo->sketch (throwing away information is usually easier than imagining it) and using that to boost the dataset for sketch->photo.

[ Parent | on HN ]

Re:The Sketchy Database: Learning to Retrieve Badly Drawn Bu... by nicklo on Saturday October 01, 2016 @08:04PM (#12620455)

Would be super cool if someone applied deep generative models to synthesize new natural images from input drawings using this dataset.

[ Parent | on HN ]

Re:The Sketchy Database: Learning to Retrieve Badly Drawn Bu... by xentripetal on Saturday October 01, 2016 @09:32PM (#12620681)

I believe one came out recently, I saw it on GitHub a few days ago. It mainly did landscape and searched for similar images then combined them to form a new one. It also supported image manipulation, eg turning a brown square purse into a green rounded one. I'll see if I can find it and edit the link in.

EDIT: https://github.com/junyanz/iGAN

Heres a good video showing it off https://www.youtube.com/watch?v=9c4z6YsBGQ0&feature=youtu.be

[ Parent | on HN ]

Re:The Sketchy Database: Learning to Retrieve Badly Drawn Bu... by sctb on Saturday October 01, 2016 @10:04PM (#12620766)

We've updated the submission title from “Photorealistic images from drawings”.

[ Parent | on HN ]