News for hackers. Stuff that matters.

A comment on Ask HN: Best practices for ethical web scraping?. Parent comment

Re:Ask HN: Best practices for ethical web scraping? by Reelin on Saturday April 04, 2020 @05:33PM (#22781719)
> As part of my learning in data science, I need/want to gather data.

Also not web scraping, but a few other public data set sources to check.

https://registry.opendata.aws

https://github.com/awesomedata/awesome-public-datasets

[ Parent | on HN ]
Re:Ask HN: Best practices for ethical web scraping? by aspyct on Saturday April 04, 2020 @06:16PM (#22781993)
Thanks!
[ Parent | on HN ]
Re:Ask HN: Best practices for ethical web scraping? by smcnally on Sunday April 05, 2020 @12:21AM (#22783715)
also https://www.reddit.com/r/datasets
[ Parent | on HN ]
Re:Ask HN: Best practices for ethical web scraping? by aspyct on Sunday April 05, 2020 @03:35AM (#22784318)
The comments that keep on giving :D
[ Parent | on HN ]
Re:Ask HN: Best practices for ethical web scraping? by analyticascent on Monday April 06, 2020 @05:58AM (#22792197)
I appreciate you starting this thread, these are great resources people are posting.

Common Crawl is the data set to master if someone wants to use the fruits of web scraping without actually doing the web scraping.

[ Parent | on HN ]