Portia, an open-source visual web scraper

Re:Portia, an open-source visual web scraper by vaviloff on Tuesday April 01, 2014 @02:21PM (#7510494)

CasperJS is a higher-level wrapper for PhantomJS, so - yes, it could be done with PhantomJS solely... But you wouldn't want to, because CasperJS makes automation easier.

[ Parent | on HN ]

Re:Portia, an open-source visual web scraper by uptown on Tuesday April 01, 2014 @02:37PM (#7510633)

Are there any libraries to facilitate database-connectivity to SQL Server or MySQL from javascript? I've used CasperJS to scrape some sites, but always fall back on post-processing the scraped data with another program in order to get it into my database. I'd love to be able to do it all from one piece of code.

[ Parent | on HN ]

Re:Portia, an open-source visual web scraper by gee_totes on Tuesday April 01, 2014 @04:56PM (#7511610)

You could always have your CasperJS scraping script make an AJAX request to a RESTful API for your MySQL DB. You won't be doing it all from one piece of code, but you'll be doing about 90% of it.

[ Parent | on HN ]