News for hackers. Stuff that matters.

A comment on Tusk – Short-lived anonymous feedback. Parent comment

Re:Tusk – Short-lived anonymous feedback by yarcob on Monday February 08, 2021 @09:09AM (#26064634)
Their docs say they do: https://developers.google.com/search/reference/robots_txt

They do say that they ignore robots.txt for user-initiated actions eg. Google Translate, which makes sense in my opinion.

[ Parent | on HN ]
Re:Tusk – Short-lived anonymous feedback by mxxx on Monday February 08, 2021 @03:14PM (#26069522)
Ah yes you’re right, my apologies. They don’t support the noindex directive inside a robots.txt any more.
[ Parent | on HN ]