News for hackers. Stuff that matters.

A comment on The Fall of Stack Overflow. Parent comment

Re:The Fall of Stack Overflow by XCSme on Tuesday July 25, 2023 @08:47AM (#36861437)
Then, maybe the "crawler" should be an actual PC navigating to the browser, taking a screenshot (or live feed) of the page and processing it with AI.
[ Parent | on HN ]
Re:The Fall of Stack Overflow by pjc50 on Tuesday July 25, 2023 @11:03AM (#36863352)
Eh, Google choose to be identifiable as googlebot and to obey robots.txt for other reasons of "good citizenship", because not everybody wants to be crawled.
[ Parent | on HN ]