Upton: A Web Scraping Framework

Re:Upton: A Web Scraping Framework by terhechte on Monday July 22, 2013 @04:30PM (#6086273)

For the past couple of years, I've always done any web scraping with trusty python + beautiful soup or elementtree. I've recently started doing it with Clojure + Enlive (mostly as an excuse to use clojure for less academic excercises) but I really like it.

From that perspective, Upton looks pretty cool, especially the debug mode.

[ Parent | on HN ]

Re:Upton: A Web Scraping Framework by dangoldin on Monday July 22, 2013 @05:22PM (#6086544)

Have you taken a look at Scrapy (http://scrapy.org/)? My evolution has been from Perl to Python and I recently did a project with Scrapy that left me pretty happy.

[ Parent | on HN ]

Re:Upton: A Web Scraping Framework by look_lookatme on Monday July 22, 2013 @08:30PM (#6087417)

Scrapy is really awesome. It's a soup to nuts queuing, fetching and extraction workflow tool and if I had to start a larger than trivial project (like an rss reader or shopping aggregation site) I would base my spider toolchain on it.

[ Parent | on HN ]

Re:Upton: A Web Scraping Framework by dangoldin on Monday July 22, 2013 @10:54PM (#6087934)

Yea - I was definitely impressed by it and I just got started. It felt as if I was able to get rid of all the boilerplate and just focus on getting the next page that needed to be crawled and the information to extract.

[ Parent | on HN ]

Re:Upton: A Web Scraping Framework by jeremybmerrill on Monday July 22, 2013 @04:54PM (#6086403)

Thanks, I hope you like it.

[ Parent | on HN ]