News for hackers. Stuff that matters.

A comment on I was wrong about robots.txt. Parent comment

Re:I was wrong about robots.txt by pimterry on Thursday July 17, 2025 @05:56AM (#44591539)
Cloudflare have some new bot verification proposals designed to fix this, with cryptographic proofs that the user-agent is who they say they are: https://blog.cloudflare.com/web-bot-auth/.
[ Parent | on HN ]
Re:I was wrong about robots.txt by EPendragon on Thursday July 17, 2025 @09:34AM (#44593227)
It is awesome to see that there are efforts to improve upon the old standards with modern web security standards
[ Parent | on HN ]