News for hackers. Stuff that matters.
Re:Cloudflare's new DNS attracting 'gigabits per second' of ... by kuschku on Friday April 06, 2018 @09:40AM (#16773103)
I agree. Part of what makes the web the web is that anyone has to be able to crawl its data, and build new technology on top.

As long as Cloudflare blocks that, it's hostile.

Maybe a way to register as spider with Cloudflare, and get a token that one passes in the header, with strict rate limits would be much better than the current solution of just showing captchas.

There are no comments on this comment yet.