Large Public Datasets

Re:Large Public Datasets by mindcrime on Saturday May 10, 2014 @11:38PM (#7727667)

It's a discussion of "good places to find large datasets open to the public". There are 132 or so answers, so I wouldn't even try to copy them all here (copyright issues aside anyway), but here's the current contents of the answer wiki. Note that some of these got partially truncated by the c&p from Quora, so if you want the full URL, you'll have to login or get somebody else to fix it. I'm too lazy for all that right now. :-)

Here are many of the links mentioned so far:

Cross-disciplinary data repositories, data collections and data search engines:

http://usgovxml.com

http://aws.amazon.com/datasets

http://databib.org

http://datacite.org

http://figshare.com

http://linkeddata.org

http://reddit.com/r/datasets

http://thedatahub.org alias http://ckan.net

http://quandl.com

Social Network Analysis Interactive Dataset Library (Social Network Datasets)

Datasets for Data Mining

http://enigma.io

Single datasets and data repositories

http://archive.ics.uci.edu/ml/

http://crawdad.org/

http://data.austintexas.gov

http://data.cityofchicago.org

http://data.govloop.com

http://data.gov.uk/

http://data.medicare.gov

http://data.seattle.gov

http://data.sfgov.org

http://data.sunlightlabs.com

https://datamarket.azure.com/

http://developer.yahoo.com/geo/g...

http://econ.worldbank.org/datasets

http://en.wikipedia.org/wiki/Wik...

http://factfinder.census.gov/ser...

http://ftp.ncbi.nih.gov/

http://gettingpastgo.socrata.com

http://googleresearch.blogspot.com

http://books.google.com/ngrams/

http://medihal.archives-ouvertes.fr

http://public.resource.org/

http://rechercheisidore.fr

http://snap.stanford.edu/data/in...

http://timetric.com/public-data/

https://wist.echo.nasa.gov/~wist...

http://www2.jpl.nasa.gov/srtm

http://www.archives.gov/research...

http://www.bls.gov/

http://www.crunchbase.com/

http://www.dartmouthatlas.org/

http://www.data.gov/

http://www.datakc.org

http://dbpedia.org

http://www.delicious.com/jbaldwi...

http://www.faa.gov/data_research/

http://www.factual.com/

http://research.stlouisfed.org/f...

http://www.freebase.com/

http://www.google.com/publicdata...

http://www.guardian.co.uk/news/d...

http://www.infochimps.com

http://www.kaggle.com/

http://build.kiva.org/

http://www.nationalarchives.gov....

http://www.nyc.gov/html/datamine...

http://www.ordnancesurvey.co.uk/...

http://www.philwhln.com/how-to-g...

http://www.imdb.com/interfaces

http://imat-relpred.yandex.ru/en...

http://www.dados.gov.pt/pt/catal...

http://knoema.com

http://daten.berlin.de/

http://www.qunb.com

http://databib.org/

http://datacite.org/

http://data.reegle.info/

http://data.wien.gv.at/

http://data.gov.bc.ca

https://pslcdatashop.web.cmu.edu/ (interaction data in learning environments)

http://www.icpsr.umich.edu/icpsrweb/CPES/ - Collaborative Psychiatric Epidemiology Surveys: (A collection of three national surveys focused on each of the major ethnic groups to study psychiatric illnesses and health services use)

[ Parent | on HN ]

Re:Large Public Datasets by eksith on Saturday May 10, 2014 @11:43PM (#7727680)

Thanks! That's actually a very helpful list. It's ironic that it's posted on a large closed dataset like Quora ;)

[ Parent | on HN ]

Re:Large Public Datasets by zwegner on Sunday May 11, 2014 @03:30AM (#7728064)

Some of the links got truncated by Quora, so here's the same list (modulo intervening wiki edits) with full URLs:

  http://usgovxml.com
  http://aws.amazon.com/datasets
  http://databib.org
  http://datacite.org
  http://figshare.com
  http://linkeddata.org
  http://reddit.com/r/datasets
  http://thedatahub.org
  http://ckan.net
  http://quandl.com
  http://www.growmeme.com/overview
  http://www.kdnuggets.com/datasets/index.html
  http://enigma.io
  http://archive.ics.uci.edu/ml/
  http://crawdad.org/
  http://data.austintexas.gov
  http://data.cityofchicago.org
  http://data.govloop.com
  http://data.gov.uk/
  http://data.medicare.gov
  http://data.seattle.gov
  http://data.sfgov.org
  http://data.sunlightlabs.com
  https://datamarket.azure.com/
  http://developer.yahoo.com/geo/geoplanet/data/
  http://econ.worldbank.org/datasets
  http://en.wikipedia.org/wiki/Wikipedia:Database_download
  http://factfinder.census.gov/servlet/DatasetMainPageServlet
  http://ftp.ncbi.nih.gov/
  http://gettingpastgo.socrata.com
  http://googleresearch.blogspot.com/2006/08/all-our-n-gram-are-belong-to-you.html
  http://books.google.com/ngrams/
  http://medihal.archives-ouvertes.fr
  http://public.resource.org/
  http://rechercheisidore.fr
  http://snap.stanford.edu/data/index.html
  http://timetric.com/public-data/
  https://wist.echo.nasa.gov/~wist/api/imswelcome/
  http://www2.jpl.nasa.gov/srtm
  http://www.archives.gov/research/alic/tools/online-databases.html
  http://www.bls.gov/
  http://www.crunchbase.com/
  http://www.dartmouthatlas.org/
  http://www.data.gov/
  http://www.datakc.org
  http://dbpedia.org
  http://www.delicious.com/jbaldwinconnect/DataSets
  http://www.faa.gov/data_research/
  http://www.factual.com/
  http://research.stlouisfed.org/fred2/
  http://www.freebase.com/
  http://www.google.com/publicdata/directory
  http://www.guardian.co.uk/news/datablog
  http://www.infochimps.com
  http://www.kaggle.com/
  http://build.kiva.org/
  http://www.nationalarchives.gov.uk/doc/open-government-licence/open-government-licence.htm
  http://www.nyc.gov/html/datamine/html/home/home.shtml
  http://www.ordnancesurvey.co.uk/oswebsite/opendata/
  http://www.philwhln.com/how-to-get-experience-working-with-large-datasets
  http://www.imdb.com/interfaces
  http://imat-relpred.yandex.ru/en/datasets
  http://www.dados.gov.pt/pt/catalogodados/catalogodados.aspx
  http://knoema.com
  http://daten.berlin.de/
  http://www.qunb.com
  http://databib.org/
  http://datacite.org/
  http://data.reegle.info/
  http://data.wien.gv.at/
  http://data.gov.bc.ca
  https://pslcdatashop.web.cmu.edu/
  http://www.icpsr.umich.edu/icpsrweb/CPES/

[ Parent | on HN ]