News for hackers. Stuff that matters.

A comment on Web Scraping with Beautiful Soup (2014).

Re:Web Scraping with Beautiful Soup (2014) by pryelluw on Friday November 11, 2016 @02:29PM (#12933556)
I enjoy using scrapy because it allows for a bit more functionality. Check it out if beautiful soup is too simplemente for your needs.
[ Parent | on HN ]
Re:Web Scraping with Beautiful Soup (2014) by hanniabu on Friday November 11, 2016 @02:32PM (#12933579)
Which would you say is easier for a beginner(to scraping and to python in general) to use?
[ Parent | on HN ]
Re:Web Scraping with Beautiful Soup (2014) by reubano on Friday November 11, 2016 @02:49PM (#12933708)
BS4 is easier but scrapy is more powerful. E.G., with scrapy you can program to follow all 'next' links and download 100s of pages in parallel. BS4 is more for grabbing content from a single page.
[ Parent | on HN ]
Re:Web Scraping with Beautiful Soup (2014) by pryelluw on Friday November 11, 2016 @07:41PM (#12935823)
Scrapy is a framework and does require knowledge of python oop and idioms. One thing is that knowing scrapy is a career boost because there is a good market out there for it.

What are your goals? Simple scraping? Then bs4 is ok. If more complex then go for scrapy. Its the django of web scrapers.

[ Parent | on HN ]