GPTBot – OpenAI’s Web Crawler

Re:GPTBot – OpenAI’s Web Crawler by LinuxBender on Monday August 07, 2023 @06:22PM (#37041763)

Google have numerous robots that do not say Googlebot in the user-agent. They look just like Android cell phones. That is how they spot malicious sites or sites that are trying to game SEO or what-not.

[ Parent | on HN ]

Re:GPTBot – OpenAI’s Web Crawler by muttled on Tuesday August 08, 2023 @11:34AM (#37050583)

I'm picturing Google Street View cars driving around with a box of Pixels in the back, connecting to open WiFi and trying sites and that's why Google can now narrow down your location from what SSIDs are available.

[ Parent | on HN ]

Re:GPTBot – OpenAI’s Web Crawler by morkalork on Tuesday August 08, 2023 @05:45PM (#37055872)

Speaking of rolling around with box of android devices:

https://www.theguardian.com/technology/2020/feb/03/berlin-ar...

Also, I would've sworn that happened circa 2015 and not 2020. The passing of time for the last few years has such a muddled feeling.

[ Parent | on HN ]