News for hackers. Stuff that matters.
Re:“Magic links” can end up in Bing search results, renderin... by swiftcoder on Monday June 27, 2022 @07:23AM (#31893028)
Isn't this just the "link preview" feature, that is enabled by default in outlook?

Many email clients generate link previews so that they can display a thumbnail of the webpage. Would seem to be necessary to filter out those referrers from the validate link

Re:“Magic links” can end up in Bing search results, renderin... by rsbadger on Monday June 27, 2022 @07:26AM (#31893059)
Essentially yes - but "previewing" link and sending that link to Bingbot to crawl and index is another matter.

Imagine you send someone a "private" link to a file...Bing sees that and indexes it for the world to see. Not cool.

Re:“Magic links” can end up in Bing search results, renderin... by swiftcoder on Tuesday June 28, 2022 @02:05AM (#31904604)
Is this actually the Bingbot crawling for the index, or do they just use the same bingbot code to generate link previews?

If I have a well-tested crawler sitting here, I don't see why I'd write a brand new one just to fetch previews in outlook...

Re:“Magic links” can end up in Bing search results, renderin... by Gigachad on Monday June 27, 2022 @07:45AM (#31893193)
“Private links” should be covered by robots.txt. The only case I see this happening is for those “anyone with link” shares and those are easy to cover.
Re:“Magic links” can end up in Bing search results, renderin... by rsbadger on Monday June 27, 2022 @07:55AM (#31893278)
AFAIK, most of those "private" links are actually just "unlisted" but they're still public. I'm sure Bing is indexing those too...
Re:“Magic links” can end up in Bing search results, renderin... by Gigachad on Monday June 27, 2022 @06:53PM (#31901983)
Anything private should ideally be put behind authentication. If that isn't possible, than robots.txt. Search engines are _meant_ to index everything that is publicly accessible and not blacklisted on robots.txt.