News for hackers. Stuff that matters.
Re:Show HN: Business.txt - Standard Proposal by novaleaf on Monday August 27, 2012 @10:16AM (#4438887)
what about meta entry(s) in the robots.txt, specifying if additional things like business.txt exist (and where)
Re:Show HN: Business.txt - Standard Proposal by cpach on Monday August 27, 2012 @11:26AM (#4439243)
But robots.txt is just for exclusion directives, right? AFAICT it's not really used for assistance with finding resources.
Re:Show HN: Business.txt - Standard Proposal by tg3 on Monday August 27, 2012 @11:36AM (#4439296)
That's true, but you can also specify the Sitemap from the robots.txt, which is an inclusion standard.