Re:What one may find in robots.txt
by psykovsky on Monday May 18, 2015 @07:00AM
(#9563698)
The websites I manage aren't Facebook or Google sized. Or even HN sized. I don't see that as a real problem at all.
[ Parent
| on HN ]