Re:GPU Open Analytics Initiative
by pplonski86 on Friday May 12, 2017 @12:23PM
(#14325336)
What is data size (how many GB) that needs such frameworks?
[ Parent
| on HN ]
Re:GPU Open Analytics Initiative
by anilshanbhag on Friday May 12, 2017 @02:24PM
(#14326471)
Anything from 10GB to 1TB works well with these systems. They can scale beyond 1TB if you use multiple machines.
The catch is GPU memory is more expensive than CPU memory. You only want to use it if: 1) Your workload is compute heavy - think machine learning OR 2) You care about speed i.e., you want queries to be interactive.
With these in-memory DBs, scaling beyond 1TB becomes prohibitively expensive for most companies.
Note that MapD is not fully in-memory. It can pull from CPU Memory and SSD as necessary. So its essentially as fast as the fastest disk-based database at its worst and likely two orders-of-magnitude or more faster in many common scenarios where certain subsets of the data are repeatedly queries (for example, BI workloads or when repeatedly pulling out subsets of a dataset to train a model)