devx_

Re:Scraping via Googlebot – How is it possible? on Sunday November 23, 2025 @03:57PM (#46027257)

Not sure how this is relevant.

Re:Scraping via Googlebot – How is it possible? on Sunday November 23, 2025 @03:57PM (#46027253)

Correct me if I'm wrong, but I believe you referring to the Rich Results Test. Fetching through that embeds `Google-InspectionTool` in the user agent, which isn't the case here.

Scraping via Googlebot – How is it possible?

Posted by devx_ on Sunday November 23, 2025 @02:08PM
from the self-post dept.

Hi,I run a website that recently experienced unusually high traffic from what appeared to be legitimate Googlebot. After investigating the access patterns, I was able to identify the source through some creative analysis.BackgroundSomeone has been scraping my website extensively using what appears to be authentic Googlebot. I traced the activity back to the person responsible, and they revealed...

( Read More... | 6 comments | 3 points | on HN )