Re:Poll: Should this site be visible to Yahoo, MSN crawlers?
by Alex3917 on Wednesday April 16, 2008 @02:02PM
(#165403)
I wonder if one could improve the efficiency of the crawlers by dynamically updating the robots.txt based on votes and time.
[ Parent
| on HN ]
There are no comments on this comment yet.