Re:Ask HN: Are you working on interesting technical problems?
by tudelo on Tuesday June 05, 2018 @10:27AM
(#17237888)
Where are you getting this weather data?
[ Parent
| on HN ]
Re:Ask HN: Are you working on interesting technical problems?
by throwawaymath on Tuesday June 05, 2018 @10:43AM
(#17238078)
All over. It's not strictly weather data, either. The majority of the data comes from programs within DSCOVR, LPDAAC, NASA, NOAA, NCDC, USGS, ARGO, CDC, CDIAC ORNL, DOE, EIA, EPA, FDA, EOSWEB, USFS, PHMSA and USDA. The snapshots I have are already around 50TB; I'm working on setting up updates at daily resolution for each of these sources, documenting it all and transforming it into something queryable.
Re:Ask HN: Are you working on interesting technical problems?
by tudelo on Tuesday June 05, 2018 @11:22AM
(#17238515)
Is there a possibility that there is just too much data for one person to handle in a meaningful way?
Re:Ask HN: Are you working on interesting technical problems?
by throwawaymath on Tuesday June 05, 2018 @11:43AM
(#17238741)
Of course, that may very well be a possibility. In that case I'll have to scale up the effort.