News for hackers. Stuff that matters.
Re:Scrape like the big boys by max002 on Friday November 05, 2021 @07:09PM (#29125704)
Its easy to detect chrome headless so scraping with it is not really how "big" boys do it :D the only scrapers/bots that are really hard to detect are the ones running and controlling real browser and not chromium. I do a lot od research aggainst abitbot systems, some times is friday night. If you spend each one in pub it doesnt mean your normal.
Re:Scrape like the big boys by beaugunderson on Saturday November 06, 2021 @07:26PM (#29134846)
puppeteer-extra and undetected-chromedriver beg to differ :)
Re:Scrape like the big boys by max002 on Sunday November 07, 2021 @02:13PM (#29142491)
Not really, i did test it (and use it for some cases), but there are still sites that detects it. I can and anyone who can check webgl renderer name, though this can be done by faking driver name but thats just one of many ways:) Its ongoing fight. If you dont move your mouse or type faster than 95% of my portal users i can detect you with js script written in under 1 minute.
Re:Scrape like the big boys by incolumitas on Saturday November 06, 2021 @06:30PM (#29134437)
:D