News for hackers. Stuff that matters.
Re:Upton: A Web Scraping Framework by baudehlo on Monday July 22, 2013 @05:32PM (#6086604)
Or do what we do at Hubdoc, and use both Node and Phantom. Node for performance where it's possible, and Phantom where the site has been built in such a way that scraping in Node becomes not worth the effort of figuring out all the weird stuff they've done in client side JS.

We maintain a Node to Phantom bridge for this: https://github.com/baudehlo/node-phantom-simple

Re:Upton: A Web Scraping Framework by kanzure on Monday July 22, 2013 @06:39PM (#6086938)
Curious, you use the webserver module in phantomjs, is that right? And that's how you do the inter-process communication? I'm curious how you chose that over websockets, or over HTTP polling from your phantomjs client against a local node server..

What about using something like node-gir, or whatever appjs does to combine the event loops of node/v8 and chromium/v8?