Re:Google launches Public Datasets program
by fhoffa on Wednesday March 30, 2016 @08:12AM
(#11388658)
Look... if you don't have problems that take hours to solve in your current environment, you haven't found big data problems (yet). And that's OK. Not everyone works with big data.
But if one day you do, please ping me, it will be fun to do a follow up.
But that's my entire point? Why are all the test datasets small enough to be faster in pgsql than the latency to Google would take?
I mean, if I wanted to showcase my software, I'd use a huge dataset with a complex problem — say protein folding, or superconductor molecular analysis — and show it in comparison on bigquery and a standard local database.
Although these two examples are bad, as I know from my university that they can be solved in the same time for less money locally than by using bigquery