Re:Cloudflare Crawl Endpoint
by memothon on Tuesday March 10, 2026 @07:18PM
(#47330012)
I've used browser rendering at work and it's quite nice. Most solutions in the crawling space are kind of scummy and designed for side-stepping robots.txt and not being a good citizen. A crawl endpoint is a very necessary addition!
[ Parent
| on HN ]
There are no comments on this comment yet.