News for hackers. Stuff that matters.
Re:Ask HN: Best practices for ethical web scraping? by mpclark on Saturday April 04, 2020 @01:22PM (#22779745)
Also, if a target site is behind Cloudflare then you probably won’t be able to masquerade as any of the popular bots - they block fake google/yandex bots.

There are no comments on this comment yet.