News for hackers. Stuff that matters.

A comment on Ask HN: Does robots.txt serve a real purpose?. Parent comment

Re:Ask HN: Does robots.txt serve a real purpose? by mgalka on Thursday March 03, 2016 @07:32PM (#11221141)
Sounds like you are speaking specifically about the sitemap. Agree that they serve a good purpose, though you don't need robots.txt to have a sitemap.
[ Parent | on HN ]

There are no comments on this comment yet.