News for hackers. Stuff that matters.
Re:It is possible to detect and block Chrome headless by pbhjpbhj on Thursday January 18, 2018 @06:51AM (#16176421)
You probably want the web equivalent of malicious compliance - an algorithmically generated web-hole or similar. That way the bot author isn't entirely sure you're on to them; it could be a bot or server error. Like send the right headers but garbage data that looks like it's compressed but isn't, or doubly compressed garbage, or trim pages at a different place (before anything interesting), or slow data transfers, or ...
Re:It is possible to detect and block Chrome headless by justrobert on Thursday January 18, 2018 @12:05PM (#16178715)
The main thing headless chrome saves is having to spawn and manage xvfb per instance.

If more sites adopt serving fake data to headless chrome, people will just return to the old xvfb workflow for those sites, and use headless for everything else.

Could even use a small set of xvfb scrapers to verify the results, and automate detection of false data.

Re:It is possible to detect and block Chrome headless by psandersen on Thursday January 18, 2018 @09:08AM (#16177081)
Just randomize the content they're trying the scrape imho
Re:It is possible to detect and block Chrome headless by dx034 on Thursday January 18, 2018 @10:07AM (#16177576)
I saw that with a page and it works extremely well. Inserting wrong data based on rules (there it was on rate limiting rather than user agent) is extremely hard to detect. The scraper never knows if they see real or wrong data.

On the other hand, this will also get wrong data in search engines.

Re:It is possible to detect and block Chrome headless by imtringued on Friday January 19, 2018 @07:12AM (#16185610)
I think you could get pretty far just by simply scrambling the CSS classes and ids.