Re:It is not possible to detect and block Chrome headless
by lr4444lr on Thursday January 18, 2018 @02:19PM
(#16180382)
Just out of curiosity, how are you going to handle Luminati? They're bot-networking home users all over the world in exchange for free VPN.
Re:It is not possible to detect and block Chrome headless
by thaumaturgy on Thursday January 18, 2018 @06:19PM
(#16182554)
That's pretty neat.
For our purposes, those residential IPs that are used maliciously through this service will just hurt the reputation of the ISP they belong to. I suspect we'll see something shake out in the data where this activity is limited to some kind of specific demographics (ISP, netrange, geographic location) and shouldn't interfere too badly with the system as a whole.
Re:It is not possible to detect and block Chrome headless
by lr4444lr on Thursday January 18, 2018 @07:29PM
(#16183008)
That's what I thought. In case you were wondering, as a Luminati user (on the proxy end, not the VPN end thank god), my experience has been higher latency and request fail rate than on traditional proxy networks. It's definitely not my first weapon in the arsenal, but since the target for multiple reasons already has an effective high latency and my users are sufficiently motivated enough to wait for what I'm delivering, it's a pretty effective last resort.
Re:It is not possible to detect and block Chrome headless
by saas_co_de on Thursday January 18, 2018 @02:43PM
(#16180616)
wow. hadn't seen this before.
Re:It is not possible to detect and block Chrome headless
by Jach on Thursday January 18, 2018 @01:44PM
(#16180004)
Maybe companies can instead focus on improving network infrastructure and software architecture to be able to eat any amount of scraper traffic since at some point it must become indistinguishable from a ddos that also needs to be handled...
Re:It is not possible to detect and block Chrome headless
by tehlike on Thursday January 18, 2018 @02:03PM
(#16180194)
Its not about ddos. Think amazon. They probably dont want to reveal all of heir inventory, their price history and so on. Even their api had rate limiting per second, and i doubts it is because of the load
It won't be very long before it'll be possible to identify most of the common proxying networks and block those.
Scrapers can respond by setting up something like an ssh tunnel from a residential high speed connection to a remote server (so that scraping run on the remote server appears to come from the residential connection) and using that as a proxy, but it adds another level of effort.
(I am generally on the scrapers' side on this and think it's ultimately futile to try to block all scrapers. Website administrators need to accept that anything they put on the internet is public and get over it.)