News for hackers. Stuff that matters.

A comment on Sourcehut will blacklist the Go module mirror. Parent comment

Re:Sourcehut will blacklist the Go module mirror by yamtaddle on Monday January 09, 2023 @02:45PM (#34315004)
Google began pushing for it to become an Internet standard—explicitly to be applicable to any URI-driven Internet system, not just the Web—in 2019, and it was adopted as an Internet standard in 2022.

https://developers.google.com/search/blog/2019/07/rep-id

[ Parent | on HN ]
Re:Sourcehut will blacklist the Go module mirror by cmatthias on Tuesday January 10, 2023 @09:48AM (#34325628)
This is true but irrelevant to the parent's question -- in the article, it's made clear that Google's requests are happening over HTTP, which is the most obvious reason why robots.txt should be respected.
[ Parent | on HN ]
Re:Sourcehut will blacklist the Go module mirror by yamtaddle on Tuesday January 10, 2023 @10:04AM (#34325875)
It's relevant because it attacks the premise of their objection.
[ Parent | on HN ]