News for hackers. Stuff that matters.

A comment on Visualizing Large Datasets on the GPU with Vega and MapD. Parent comment

Re:Visualizing Large Datasets on the GPU with Vega and MapD by tmostak on Monday July 24, 2017 @10:30PM (#14844625)
MapD customers typically run our product on multiple servers with multiple GPUs per node. So 4 servers with 8 Nvidia P40s each has 4X192GB = 768GB of VRAM. Note MapD compresses data and also keeps data in CPU RAM as needed. Even two servers with these GPUs or 4 servers with gamer GPUs is enough to query and visualize an 11B record shipping dataset without a hitch (https://www.mapd.com/demos/ships), a demo running on four servers with 8 Nvidia 1080 Tis each.

Other customers with smaller datasets (i.e. less than a few hundred million records) are able to run with a single GPU.

We're not going after petabyte size datasets (where <100ms querying is rarely important), so ability to scale has rarely been an issue.

[ Parent | on HN ]
Re:Visualizing Large Datasets on the GPU with Vega and MapD by qeternity on Tuesday July 25, 2017 @03:57AM (#14845965)
I would love to see some comparisons to other mpp in memory databases as, and I mean this with all due respect, it's difficult to gauge what the impact of GPUs are. Have you benchmarked against anything like Memsql? Also, do you mind if I ask where you got the ship data from?
[ Parent | on HN ]
Re:Visualizing Large Datasets on the GPU with Vega and MapD by ethikal on Tuesday July 25, 2017 @09:10AM (#14847404)
For some benchmarks, take a look here: http://tech.marksblogg.com/benchmarks.html
[ Parent | on HN ]
Re:Visualizing Large Datasets on the GPU with Vega and MapD by tmostak on Monday July 24, 2017 @10:33PM (#14844637)
Note that although we can run on CPU, CPUs do not have the graphics pipeline and the memory bandwidth necessary for interactive server-side visualizations like this.
[ Parent | on HN ]
Re:Visualizing Large Datasets on the GPU with Vega and MapD by coherentpony on Tuesday July 25, 2017 @01:22AM (#14845283)
Cool, thanks for the explanation.
[ Parent | on HN ]
Re:Visualizing Large Datasets on the GPU with Vega and MapD by dogma1138 on Tuesday July 25, 2017 @04:05AM (#14846015)
Do you use Pascal's unified memory for over-provisioning? Since Maxwell only supports unified virtual memory addresses with the host upto the VRAM limit.
[ Parent | on HN ]
Re:Visualizing Large Datasets on the GPU with Vega and MapD by tmostak on Tuesday July 25, 2017 @10:18AM (#14847865)
MapD predates good virtual (unified) memory support on GPUs and so we built our own caching mechanism where each GPU has its own buffer pool and pulls from a CPU buffer pool (i.e a network of buffer pools). This approach still has the advantage of giving us a lot of control of where we put data, how much we leave for other processes, as well as allowing us to pin data in VRAM on a specific GPU.
[ Parent | on HN ]