News for hackers. Stuff that matters.
Re:Remind HN: Go to Noobstories, flag spam by thinkbohemian on Sunday September 05, 2010 @10:20PM (#1665658)
Robots.txt

User-Agent: * Disallow: /x? Disallow: /vote? Disallow: /reply? Disallow: /submitted? Disallow: /submitlink? Disallow: /threads?

Re:Remind HN: Go to Noobstories, flag spam by sorbus on Sunday September 05, 2010 @10:25PM (#1665664)
One assumes that a core characteristic of spammers and spambots is that they ignore Robots.txt - or at least I would be extremely surprised if disallowing them in robots.txt has any effect.