News for hackers. Stuff that matters.

A comment on Scraping via Googlebot – How is it possible?.

Re:Scraping via Googlebot – How is it possible? by cmckn on Sunday November 23, 2025 @02:52PM (#46026717)
The search console for domains allows you to put in a URL and test-scrape it to see how things look to the bot. Could be some reverse-engineering/abuse of that API.
[ Parent | on HN ]
Re:Scraping via Googlebot – How is it possible? by devx_ on Sunday November 23, 2025 @03:57PM (#46027253)
Correct me if I'm wrong, but I believe you referring to the Rich Results Test. Fetching through that embeds `Google-InspectionTool` in the user agent, which isn't the case here.
[ Parent | on HN ]