Re:OpenAI Announces SearchGPT
by jsheard on Thursday July 25, 2024 @02:58PM
(#41072126)
Google actually provides means of validating whether a request really came from them, so masquerading as Googlebot would probably backfire on you. I would expect the big CDNs to flag your IP address as malicious if you fail that check.
https://developers.google.com/search/docs/crawling-indexing/...