Re:Analyzing One Million Robots.txt Files
by mindB on Sunday December 24, 2017 @01:29PM
(#16000608)
History presented in this post was very interesting, but the analysis ended up disappointing. The article ends just after they had managed to narrow their sample of robots.txt files to exclude duplicate and derivative files. They don't even present any summary statistics for this filtered sample.
[ Parent
| on HN ]
There are no comments on this comment yet.