Re:GPTBot – OpenAI’s Web Crawler
by mdaniel on Monday August 07, 2023 @11:56AM
(#37036168)
If they were going to the amount of energy to do a 2nd crawl using a different user agent, then why bother advertising the user agent at all and just feed it the Chrome one like every other home-grown spider does
[ Parent
| on HN ]
There are no comments on this comment yet.