Re:Avoiding bot detection: How to scrape the web without get...
by hattmall on Monday November 01, 2021 @12:33AM
(#29063019)
403 is per request, not requester. I get random 403s when just browsing some websites. Does that mean I should close the browser and not hit refresh for fear of breaking some wire fraud unauthorized access law?
Re:Avoiding bot detection: How to scrape the web without get...
by marginalia_nu on Monday November 01, 2021 @04:55AM
(#29064351)
If you go by the semantics of what the 403 code means, absolutely, that's excatly what the status code means.
In practice there's of course nuance, like anyone will occasionally type in the wrong password on a log-in screen, maybe try again and then realize it was the wrong log-in prompt. That's mostly fine.
That's different from deliberate trying to circumvent a measure like this. If you are doing the stuff in the link, you are absolutely crossing a line and you know it.
There's a large difference between "I got a 403 so I hit F5 once" and "I got a 403 so I used a residential proxy and spoofed my user-agent".
> The client does not have access rights to the content; that is, it is unauthorized, so the server is refusing to give the requested resource.
Machines don't have any legal responsibility, bot-operators do. Which is why respecting these things is sort of important. At any rate, 40x does not mean "try again with a different user agent and another IP"