News for hackers. Stuff that matters.
Re:Microsoft confirms Reddit blocked Bing Search by IvanAchlaqullah on Thursday July 25, 2024 @12:05AM (#41064598)
At this rate, robots.txt will be ignored by bots if it's not already.

Just like ads company completely ignoring Do Not Track because it's always set to deny / opt-out by the browser. It's already been fixed, but the damages is already done.

https://en.wikipedia.org/wiki/Do_Not_Track#Internet_Explorer...

Re:Microsoft confirms Reddit blocked Bing Search by whalesalad on Thursday July 25, 2024 @12:13AM (#41064637)
It’s ignored by many for years. SEM Rush and ironically Bing are an egregious violators of this in my experience.