Re:Ask HN: Best practices for ethical web scraping?
by mpclark on Saturday April 04, 2020 @01:22PM
(#22779745)
Also, if a target site is behind Cloudflare then you probably won’t be able to masquerade as any of the popular bots - they block fake google/yandex bots.
[ Parent
| on HN ]
There are no comments on this comment yet.