News for hackers. Stuff that matters.
Re:AI-Shunning robots.txt by cabirum on Thursday March 28, 2024 @09:08AM (#39850858)
Search engines are not the beacons of holiness - they sell ads, they sell data on who searched what, they manipulate results.

Search engines and AI things are typically owned by the same company. AIs are fed with the data collected by a search engine. The only difference is whether AI gets the data in realtime or waits for the search engine to collect another data dump.

Fighting windmills as I see it.

Re:AI-Shunning robots.txt by nunez on Thursday March 28, 2024 @10:00AM (#39851573)
Search engines manipulate results way less aggressively than LLMs do.