News for hackers. Stuff that matters.

A comment on Google’s robots.txt parser is now open source.

Re:Google’s robots.txt parser is now open source by rasmi on Monday July 01, 2019 @10:26AM (#20325524)
Code here: https://github.com/google/robotstxt
[ Parent | on HN ]
Re:Google’s robots.txt parser is now open source by glenneroo on Monday July 01, 2019 @12:54PM (#20327280)
It's also the 5th link "open sourced" in the article.
[ Parent | on HN ]