News for hackers. Stuff that matters.
Re:Digg banned from Google? by kiallmacinnes on Wednesday March 20, 2013 @12:28PM (#5408259)
http://digg.com/robots.txt <-- 404 Not Found

I know a robots.txt 404 shouldn't de-list you, but I would have expected digg to have one?

Maybe they requested de-listing and removed their robots.txt? Who knows!

Re:Digg banned from Google? by davenseo on Wednesday March 20, 2013 @12:58PM (#5408514)
http://digg.com/robots.txt looks fine too me
Re:Digg banned from Google? by randomdata on Wednesday March 20, 2013 @01:00PM (#5408528)
There is one there now:

  User-agent: *
  Disallow:
Re:Digg banned from Google? by kiallmacinnes on Wednesday March 20, 2013 @02:22PM (#5409140)
Yup - It's back now.. But it 100% didn't exist when I posted earlier :)