News for hackers. Stuff that matters.
Re:Google removed our website from search because it uses Yo... by blinding-streak on Monday September 19, 2022 @07:01AM (#32896704)
Robots.txt is a thing.
Re:Google removed our website from search because it uses Yo... by senko on Monday September 19, 2022 @07:08AM (#32896767)
If robots.txt were a thing, Google would never have built its search engine. Everyone would just allow Altavista and disallow everyone else, like they do with Googlebot and Bingbot now.

Also, the issue here is not that Google banned their tool from accessing YouTube because the tool didn't obey robots.txt. The issue is that Google, the search engine, nuked their site, because they didn't like its content.