News for hackers. Stuff that matters.
Re:Show HN: Robots.txt Checker (CoffeeScript, Node) w. Bookm... by alphadown on Friday October 28, 2011 @07:17AM (#3167806)
How about a robot.txt generator. Maybe it shows the user all the directories of the website (reads the sitemap) and then allows the user to disallow specific documents and directories in a point and click fashion. The generator then writes it in the proper syntax.

Yea? Nay?

There are no comments on this comment yet.