Scraping made easy with jQuery and SelectorGadget (and Node.js!)

Re:Scraping made easy with jQuery and SelectorGadget (and No... by richcollins on Saturday January 22, 2011 @01:08AM (#2130071)

I've been doing a lot of scraping in node. I've had much better luck using YUI + jsdom than jquery +jsdom. Many pages would fail using jquery and it also leaked memory like crazy.

[ Parent | on HN ]

Re:Scraping made easy with jQuery and SelectorGadget (and No... by chrisohara on Saturday January 22, 2011 @08:51AM (#2130390)

I suggest taking a look at node.io (https://github.com/chriso/node.io) - a scraping framework written for NodeJS. It uses htmlParser rather than jsdom and scales nicely. It also has support for handling timeouts, retries, etc.

[ Parent | on HN ]

Re:Scraping made easy with jQuery and SelectorGadget (and No... by Charuru on Saturday January 22, 2011 @02:42AM (#2130142)

Hmm why would this be?

[ Parent | on HN ]