News for hackers. Stuff that matters.
Re:Ask HN: How to Web Scrape in 2020? by jamil7 on Monday March 30, 2020 @05:33AM (#22726424)
Depending on what you're scraping you might run into a fair few JS-Only websites that are a pain to scrape. On top of all the things mentioned here you will need to run pages through a headless browser like puppeteer. For these sites you maybe be able to reverse engineer their APIs and attempt to scrape those rather than the pages themselves.

There are no comments on this comment yet.