Re:The State of Web Scraping 2022
by tomschwiha on Wednesday January 12, 2022 @04:15PM
(#29912768)
I believe BingPreview is acting like a regular user - so it is not behaving like a Robot but like a user.
[ Parent
| on HN ]
What does that mean exactly? An actual user can't be involved because the links that trigger a GET simply aren't there anymore. Therefore I assume it's a bot hitting faulty links it finds in its cache.