Re:Ask HN: Should you block web scrapers?
by jorgecurio on Monday February 01, 2016 @10:43PM
(#11017532)
if there was a public API that you could cross reference IP from your website that you suspect as a bot, would you use it? Would you pay for that kind of API?
Of course I would start off by ironically scraping proxies from public domain as those are often used. Not sure what else determined scrapers use to get data, but it's safe to say that 98% of scrapers are trying to pay as little money as possible and often complain when a scraper gets blocked...fuck these people seriously
Of course I would start off by ironically scraping proxies from public domain as those are often used. Not sure what else determined scrapers use to get data, but it's safe to say that 98% of scrapers are trying to pay as little money as possible and often complain when a scraper gets blocked...fuck these people seriously