Re:Turn any website into a live, structured data feed
by xnx on Thursday February 05, 2026 @01:55PM
(#46903403)
That's what robots.txt is for.
[ Parent
| on HN ]
There are no comments on this comment yet.