News for hackers. Stuff that matters.

A comment on Are You a Robot? Introducing “No CAPTCHA ReCAPTCHA”. Parent comment

Re:Are You a Robot? Introducing “No CAPTCHA ReCAPTCHA” by ChuckMcM on Wednesday December 03, 2014 @02:53PM (#8695968) 

   > if you framed the issue as a business model one, not 
   > a technical one, it might be a useful exercise.
That was kind of my point. Clearly most of the bots are trying to scrape my search engine for some specific data. I would (generally) be happy to just sell them that data rather than have them waste time trying to scrape us (that is the business model, which goes something like "Hey we have a copy of the big chunk of the web on our servers, what do you want to know?" but none of the bot writers seem willing to got there. They don't even send an email to ask us "Hey, could we get a list of every site you've crawled that uses the following Wordpress theme?" No instead they send query after query for "/theme/xxx" p=1, p=2, ... p=300.

On a good day I just ban their IP for a while, when I'm feeling annoyed I send them results back that are bogus. But the weird thing is you can't even start a conversation with these folks, and I suppose that would be like looters saying "Well ok how about you help load this on a truck for me for 10 cents on the dollar and then your store won't be damaged." or something.

[ Parent | on HN ]
Re:Are You a Robot? Introducing “No CAPTCHA ReCAPTCHA” by dennisgorelik on Saturday December 06, 2014 @12:01AM (#8708414)
You may try to contact scrapers through access denied page.

Did you try to explicitly state that your data is available for sale when denying access to p=300?

[ Parent | on HN ]
Re:Are You a Robot? Introducing “No CAPTCHA ReCAPTCHA” by SiVal on Wednesday December 03, 2014 @03:59PM (#8696308)
If you wanted to buy data from Google, how would you email? What is Google's email address?
[ Parent | on HN ]
Re:Are You a Robot? Introducing “No CAPTCHA ReCAPTCHA” by ChuckMcM on Wednesday December 03, 2014 @05:41PM (#8696818)
Google posts lots of contact information on their contact page. You would probably want to reach business development. I don't think they are willing to sell access to that index however, we (at Blekko) would. I suppose you could also try to pull it out of common crawl.
[ Parent | on HN ]