News for hackers. Stuff that matters.
Re:“Magic links” can end up in Bing search results, renderin... by snowwrestler on Tuesday June 28, 2022 @10:46AM (#31908364)
You are correct: a bot that is well-behaved (follows robots.txt directions) will not crawl your site if your robots.txt forbids crawling.

There are no comments on this comment yet.