Re:Show HN: Scrape the web by injecting JavaScript into web ...
by willu on Tuesday December 23, 2014 @09:35AM
(#8788224)
If you are able to crawl that volume as cheaply as you say, you should definitely be offering it as a service.
[ Parent
| on HN ]
Re:Show HN: Scrape the web by injecting JavaScript into web ...
by jalfresi on Tuesday December 23, 2014 @09:43AM
(#8788258)
Its a very single purpose system, so probably not amenable to general purpose crawling (though we do have a separate system that is based of the design of our core system that is a general purpose web crawler/indexer)