Re:Ask HN: How do you guys stop scrapers from mirroring your...
by bartonfink on Sunday January 16, 2011 @12:20AM
(#2108755)
Isn't this a problem for a CAPTCHA?
[ Parent
| on HN ]
Re:Ask HN: How do you guys stop scrapers from mirroring your...
by jackowayed on Sunday January 16, 2011 @02:05AM
(#2108904)
No.
If the content is only accessible after you fill out a CAPTCHA, Google won't be able to index your content, and you won't have any visitors for scrapers to steal from you.
I suppose you could provide the content without a CAPTCHA if the user agent matches the crawlers of Google, Bing, etc. But it's still a bad idea because a lot (most?) of your users will bounce on the CAPTCHA, killing your ad revenue.
Re:Ask HN: How do you guys stop scrapers from mirroring your...
by damncabbage on Sunday January 16, 2011 @08:38PM
(#2110898)
... And it's very easy to just impersonate the GoogleBot by using its User-Agent.
Re:Ask HN: How do you guys stop scrapers from mirroring your...
by sursani on Monday January 17, 2011 @02:37PM
(#2113133)
Ditto.. there is no point trying to prevent certain bots, etc from scraping your content. It's a never ending battle. Just like all the others said before me, if it's your content, you should be fine.