Re:The big data disaster
by johnrgrace on Thursday November 22, 2012 @12:05PM
(#4819346)
I always define big data as, too big for excel spreadsheets.
[ Parent
| on HN ]
There is MPI add-on for Excel, which allow to work with huge spreadsheets on HPC clusters, developed originally for some Swiss Investment bank. So Excel does scale.
I don't like that definition because Excel as software hasn't scaled with computer hardware. Excel isn't a database, and isn't designed to be one.
To put it into perspective, people were working with 1M+ record datasets in the 1990s. I tried loading a 300k record file in Excel 2010 the other day and it crashed.