News for hackers. Stuff that matters.

A comment on RoboBrowser: Your friendly neighborhood web scraper.

Re:RoboBrowser: Your friendly neighborhood web scraper by pcr0 on Thursday May 12, 2016 @02:58AM (#11681838)
Hmm, I can see why I'd want to use this library over piecing together requests and BS4 myself for every project. I love how simple the examples look.

I have a project I'm working on that will involve scraping many different websites on a daily basis. My only scraping experience so far is using cheerio[0] to scrape a single page with a 1,000 row HTML table. Should I start with something BS-based like this or should I jump straight into Scrapy? Or are there any other alternatives I should try?

[0]: https://github.com/cheeriojs/cheerio

[ Parent | on HN ]
Re:RoboBrowser: Your friendly neighborhood web scraper by jakubbalada on Thursday May 12, 2016 @05:27AM (#11682286)
If you want to scrape many websites on a daily basis, have a look at https://www.apifier.com as an alternative.

Disclaimer: I'm a cofounder there

[ Parent | on HN ]