News for hackers. Stuff that matters.

A comment on How Telegram Messenger circumvents Google Translate's API. Parent comment

Re:How Telegram Messenger circumvents Google Translate's API by inportb on Friday December 31, 2021 @09:42AM (#29748442)
> this would be like a crawler ignoring robots.txt

Google ignores the noindex directive in robots.txt now. You're supposed to put it in your HTTP response headers or HTML meta tags...

[ Parent | on HN ]
Re:How Telegram Messenger circumvents Google Translate's API by ldjb on Friday December 31, 2021 @10:24AM (#29748809)
`noindex` was a Google-specific rule that was never officially documented nor supported. I think they were perfectly entitled to withdraw support for it, especially considering there are alternatives.

https://developers.google.com/search/blog/2019/07/a-note-on-...

[ Parent | on HN ]