Re:Python web scraping
by Jake232 on Tuesday March 11, 2014 @12:18AM
(#7377396)
Thanks for the mechanize link, I'll add a note about it.
Mechanize is awfully slow though, if you need to crawl quickly it's not asynchronous. I wouldn't want to use it for general crawling. I guess you could patch the stdlib with gevent, and try and get something working that way.
Mechanize is awfully slow though, if you need to crawl quickly it's not asynchronous. I wouldn't want to use it for general crawling. I guess you could patch the stdlib with gevent, and try and get something working that way.