Crawl a website with scrapy and store extracted results with MongoDB

Re:Crawl a website with scrapy and store extracted results w... by jat1 on Monday April 23, 2012 @11:46AM (#3879337)

Also check this out for a pretty good discussion on scraping http://pyvideo.org/video/609/web-scraping-reliably-and-effic...

[ Parent | on HN ]

Re:Crawl a website with scrapy and store extracted results w... by BaltoRouberol on Monday April 23, 2012 @11:59AM (#3879420)

Yeah, I actually learnt scraping from Asheesh :) He's awesome.

[ Parent | on HN ]

Re:Crawl a website with scrapy and store extracted results w... by jat1 on Monday April 23, 2012 @12:30PM (#3879621)

I have been playing with scraping for quite some time now and have my own scripts and stuff, but I found that video informing and there were a few useful snippets I had missed.

Keep meaning to check out more of the Pycon vids

[ Parent | on HN ]