News for hackers. Stuff that matters.
Re:Ask HN: What are best tools for web scraping? by harperlee on Tuesday November 14, 2017 @04:33PM (#15699354)
I've had mixed results with scrapy, probably more based in my inexperience than other thing, but for example retrieving a posting in idealista.com with vanilla scrapy begets an error page whereas a basic wget command retrieves the correct page.

So the learning curve for simple things makes me jump to bash scripts; scrapy might prove more valuable when your project starts to scale.

But also of course: normally the best tool is the one you already know!

There are no comments on this comment yet.