Re:ROBOTS.TXT is a suicide note
by Artemis2 on Thursday January 12, 2017 @04:01AM
(#13380867)
On the page: "While the onslaught of some social media hoo-hah will demolish some servers in the modern era, normal single or multi-thread use of a site will not cause trouble, unless there's a server misconfiguration"
[ Parent
| on HN ]
There are no comments on this comment yet.