News for hackers. Stuff that matters.
Re:Ask HN: What are the best tools for web scraping for non-... by Ihateusernames2 on Wednesday November 15, 2017 @06:04AM (#15702927)
Chrome Devtools.

That sounds silly, and it is. But really, in many cases the data you want to scrape is already loaded to the page in a useful format. For example, take Politico's 2016 election results data: https://www.politico.com/2016-election/results/map/president...

If you go to devtools > network > XHR, you'll see the page for Colorado loads a file containing all of the results in a format you should be able to parse. This example is a bit tricky, as parsing it as a non-programmer might still be a bit challenging, but some sites will load data as a CSV.

I know this doesn't exactly answer your question, but I think it would help non-programmers to know this.

There are no comments on this comment yet.