Re:Google's robots.txt
by techaddict009 on Saturday February 22, 2014 @11:47PM
(#7284966)
Checkout youtube.com/robots.txt !
[ Parent
| on HN ]
For the lazy: http://www.youtube.com/robots.txt
Re:Google's robots.txt
by Mindless2112 on Sunday February 23, 2014 @01:41AM
(#7285164)
And for those who don't know the reference: http://www.youtube.com/watch?v=WGoi1MSGu64