Re:Ask HN: What are best tools for web scraping?
by djtriptych on Tuesday November 14, 2017 @11:52AM
(#15696702)
All great advice. I've written dozens of small purpose-built scrapers and I love your last point.
It's pretty much always a great idea to completely separate the parts that perform the HTTP fetches and the part that figures out what those payloads mean.
It's pretty much always a great idea to completely separate the parts that perform the HTTP fetches and the part that figures out what those payloads mean.