News for hackers. Stuff that matters.

A comment on Ask HN: I want to crawl every plain HTML website. Where do I begin?.

Re:Ask HN: I want to crawl every plain HTML website. Where d... by deepsy on Thursday December 01, 2022 @07:22PM (#33825092)
I'd probably look into AWS Lambda as you can query in parallel from different IPs at a scale cheaply.
[ Parent | on HN ]

There are no comments on this comment yet.