News for hackers. Stuff that matters.
Re:Claude can now search the web by otikik on Thursday March 20, 2025 @01:17PM (#43426014)
Apparently, the regular search crawler does it, but the ai thingie doesn't.
Re:Claude can now search the web by lucgagan on Thursday March 20, 2025 @01:20PM (#43426052)
Can confirm. My website is flooded with AI bots despite attempts to block crawlers to certain parts of it.
Re:Claude can now search the web by supriyo-biswas on Thursday March 20, 2025 @01:19PM (#43426047)
Huh? You can add Google-Extended[1] to opt out from Generative AI summaries.

[1] https://blog.google/technology/ai/an-update-on-web-publisher...

Re:Claude can now search the web by micromacrofoot on Thursday March 20, 2025 @01:41PM (#43426343)
Google will still scrape it for training data either way, this only impacts search results.
Re:Claude can now search the web by supriyo-biswas on Thursday March 20, 2025 @02:17PM (#43426908)
> Today we’re announcing Google-Extended, a new control that web publishers can use to manage whether their sites help *improve Bard and Vertex AI generative APIs*, including future generations of models that power those products.
Re:Claude can now search the web by micromacrofoot on Thursday March 20, 2025 @03:57PM (#43428126)
https://www.theverge.com/news/630079/openai-google-copyright...

they're literally asking to break laws to train AI for national security. A sentence in a press release from 2 years ago is worthless... look at what they're actually doing