Do you rely on an outside source for picture identification? If so, what's your plan if/when it fails? If not, do you plan to build your own in-house system?

Have you tested whether your code has any impact in page load speed (Y-Slow and company)?

Related to Q1, if you have your own in-house system, will you have your own database of pictures for feature extraction/matching, and how do you plan to make it large enough (and grow it fast enough) to accomodate the wide variety of pictures that people take and post?

I think that if you implement this well, you potentially have a real winner here. Best of luck to you!

We've used the Amazon product API to build our databsae, there are ~80m products in there. We're starting one vertical at a time (the algo works best for highly textured products such as watches/book/cd covers so we've started there). We build an in-memory index of image-descriptors which is queried when trying to match a product. As we start seeing other product type, we'll expand beyond amazon.

As for scaling, the index is the one component that isn't trivial to parallelise. Once we get to a stage where a single box can't store it in memory, the first step would be to shard it, with different indexes per product category. Query images will be run through a product-categorisation algo to determine which indexes they need to be run against. This is how we think we'll approach this, from previous experience, but will investigate properly (if) we get large enough.

In terms of affecting page load, our plugin will load async, as will the results returned from the image matching algorithm, so it should be fairly negligible impact.

But like any ad network, how do you guys plan on keeping up with splogs (spam blogs) abusing your service ?

What about mobile?

Cool idea.

Maybe answering them publically would get some feedback?

Best of luck.

You perhaps will find yourself where you are in the application. By the way, the idea is cool. But does your algorithms advanced enough to distinguish between items that can be bought (commercials) compared to those can't (human face, nature photographs etc to example a few)?

Here is an example of what the ads would look like on our site dump.ly:

http://dump.ly/y3uyGE2hpn2j