News for hackers. Stuff that matters.
Re:Ask HN: Anyone here do Web mining/harvesting/scraping? by truiu on Saturday October 17, 2009 @10:25AM (#886949)
I have done some webscraping with twisted and BeautifulSoup. But it was just for a few small sites.

That was when I found http://scrapy.org/ , a complete python framework for writing webcrawlers. It looks promissing, but I haven't used it so far.

There are no comments on this comment yet.