News for hackers. Stuff that matters.
Re:Cloudflare Crawl Endpoint by memothon on Tuesday March 10, 2026 @07:18PM (#47330012)
I've used browser rendering at work and it's quite nice. Most solutions in the crawling space are kind of scummy and designed for side-stepping robots.txt and not being a good citizen. A crawl endpoint is a very necessary addition!

There are no comments on this comment yet.