News for hackers. Stuff that matters.

A comment on “Magic links” can end up in Bing search results, rendering them useless. Parent comment

Re:“Magic links” can end up in Bing search results, renderin... by Xylakant on Monday June 27, 2022 @08:17AM (#31893473)
But there is nothing to indicate either in the post or in the referenced SO thread that the URLs are published to the search results. They are visited by bingbot, that much seems confirmed, but there’s no example where one of these results shows up in the public search results.
[ Parent | on HN ]
Re:“Magic links” can end up in Bing search results, renderin... by rsbadger on Monday June 27, 2022 @08:33AM (#31893656)
They were indexed in Bing results, I’ve shared the URL of that in this thread
[ Parent | on HN ]
Re:“Magic links” can end up in Bing search results, renderin... by cma on Monday June 27, 2022 @11:25AM (#31895889)
Holy shit thats bad. Do unlisted youtube and gdrive share links get indexed through this?
[ Parent | on HN ]
Re:“Magic links” can end up in Bing search results, renderin... by 9dev on Monday June 27, 2022 @01:52PM (#31898125)
You’d assume those have proper robots.txt configuration?
[ Parent | on HN ]
Re:“Magic links” can end up in Bing search results, renderin... by boxed on Monday June 27, 2022 @04:57PM (#31900695)
I have a disallow all robots.txt for a production system. Have had from the beginning.

Bing indexes it. This is my first major security incident and I have no idea how to fix this without making everything totally shitty for the users.

[ Parent | on HN ]
Re:“Magic links” can end up in Bing search results, renderin... by efreak on Thursday June 30, 2022 @09:25AM (#31932069)
Some services ignore global disallow, but will respect rules explicitly targeted at them.
[ Parent | on HN ]
Re:“Magic links” can end up in Bing search results, renderin... by boxed on Thursday June 30, 2022 @03:07PM (#31937074)
I've put in a hard block for all crawlers on all pages. Works for my scenario I think. Hopefully they don't lie in their user agent. Then it's going to be really bad.
[ Parent | on HN ]