I tend to use scrapy when doing python scraping but its a massive setup, this will be good for smaller projects.