I used Scrapy extensively last term in my non-CS co-op. Really loved how quickly I could make beautiful dashboards with a bit of Python / VBA.

My problem with Scrapy was that there was a lot of manual work involved for each site, but perhaps that speaks more about my proficiency rather than Scrapy.

What's the next step from here in terms of gathering, cleaning, and analyzing data?