News for hackers. Stuff that matters.
Re:Detecting Chrome headless by dsacco on Saturday August 05, 2017 @02:45PM (#14937214)
To be honest (and speaking from experience), the only thing that would actually make life hard for a scraper is the last suggestion. I've never personally dealt with that, which is the only reason why I concede it.

Machine generated HTML is a pain, but I wouldn't call that "very difficult" - it's more like "annoying" in the same way that having to parse HTML instead of finding a neat JSON endpoint is.

And I'm not sure how randomly inserted HTML elements would help - if you're already parsing the HTML, you can extract the relevant data. Unless you're trying to parse the HTML with regex, in which case: https://stackoverflow.com/questions/1732348/regex-match-open...

Re:Detecting Chrome headless by ivanhoe on Sunday August 06, 2017 @05:49PM (#14943805)
The first two bullets combined can make it hard to parse the page reliably, but it's also making very hard the maintenance and using of any 'good' javascript and/or css too. Not worth the trouble usually.