News for hackers. Stuff that matters.
Re:“Magic links” can end up in Bing search results, renderin... by rsbadger on Monday June 27, 2022 @07:43AM (#31893173)
Re:“Magic links” can end up in Bing search results, renderin... by scandinavian on Monday June 27, 2022 @08:24AM (#31893552)
The URL in the search is this: https://shoprocket.io/email-confirmation/34b35b1...

I don't see that in the robots.txt https://shoprocket.io/robots.txt

User-agent: *

Disallow: /cdn-cgi/l/email-protection

Disallow: /login

Disallow: /register

Disallow: /404

Am I missing something?

Re:“Magic links” can end up in Bing search results, renderin... by rsbadger on Monday June 27, 2022 @08:39AM (#31893708)
You may be seeing a stale version, try this: https://shoprocket.io/robots.txt?bypass=1

(I made a lot of changes today when testing all, including "visit as Bingbot" from their webmaster tools with and without the URL blocked by robots.txt)