Re:Artoo, the client-side scraping companion
by fiatjaf on Tuesday June 17, 2014 @05:19PM
(#7907063)
Well, I see you already have almost all I suggested. Now I would want something to make the ajaxSpider render the pages using the browser engine, instead of just getting pure HTML.
Re:Artoo, the client-side scraping companion
by Yomguithereal on Tuesday June 17, 2014 @05:26PM
(#7907112)
This is an interesting point. I created an issue on the github repository concerning this matter. Maybe you'd like to comment on it about your use case so we can improve the tool?
I don't know if it is possible, but could this run as a Chrome Extension, in a background script, loading various pages, executing code on then and keep going, storing the data at the extension's localStorage?
It could also store the code of the scrapers, for reusing.