Re:Show HN: Chromeless – Headless Chrome Automation on AWS L...
by afandian on Thursday July 27, 2017 @05:34AM
(#14863772)
What do you mean by off-brand scraping? You mean search engines that you haven't heard of, or copyright violating orgs?
Re:Show HN: Chromeless – Headless Chrome Automation on AWS L...
by pravda on Thursday July 27, 2017 @02:07PM
(#14867864)
Some AWS visitors:
Cliqzbot, VidibleScraper/1.0, CheckMarkNetwork, CCBot/2.0 (http://commoncrawl.org/faq/), linkdexbot/2.2; +http://www.linkdex.com/bots/
That last one is a "SEO platform".
And then all the off-brand scraping companies use amazonaws.com.