News for hackers. Stuff that matters.
Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots by naikrovek on Thursday October 24, 2024 @09:11AM (#41935156)
> no human looks at robots.txt

of course people look at this. it's not an everyday thing for the prototypical web user, but some of us look at those a lot.

Re:Ask HN: Website with 6^16 subpages and 80k+ daily bots by pkaeding on Thursday October 24, 2024 @11:46AM (#41936696)
Yes, but those of us also often have trouble solving captchas.