Furthermore, all that memory bandwidth is calculated against all the cores. So you have to be VERY careful in usage patterns (it doesn't work like a giant CPU). Not to mention how much it costs involved per GB or TB!
I have some experience in this area, and where GPU's & database really shine is building (and especially) re-building indexes.
Running a database on GPUs isn't going to replace all DBs overnight. But, the hardware does have it's uses; just like improved SIMD on CPU's and NVMe storage, plus developments in networking - Basically look where Intel is going since we're coming to the limits of silicon transistors..
How relevant is that when you're looking at multi-TB data sets that don't fit into computer RAM? Sure, the RAM <---> CPU bandwidth may be very wide, but the SSD connects to the computer over the same PCIe bus.
And also: when did you have this conversation? GPU performance has changed very much year by year, so what wouldn't have been worth it 2 years ago might be a huge gain now.
It's not about GPU performance, it's about the latency and bandwidth of getting that dat to the GPU. If once you ship data to the GPU, you reuse it many times for many calculations, that cost is amortized and it doesn't matter as much. But if you ship data to the GPU and use it once, then that cost will probably not be amortized. I think of databases tending to fit in the latter category.
Outside of CPU >> GPU, I'm not sure what other data movement you could be talking about. A SAS HBA or Ethernet NIC or Infiniband HBA are almost always going to be operating over the same PCIe bus the GPU uses. In the rare instances they're built onto the CPU, the "network link" is likely going to still be significantly slower than the fastest PCIe slot.
NVLink has 80GB/s [1]. DDR4 quad channel (Xeon Servers) has ~120GB/s [2]. So no this rationalization doesn't fall apart. Furthermore in the event NVLink gets faster then RAM, then you'll still be bottlenecked by RAM access, as you'll buffer here.
This of course is ignoring weird systems where you attempt to maintain ACID coherence of tables between GPU, CPU, and Disk memory. But then GPU memory size become inherently limiting as even the biggest max out at ~32GB.
[1] https://en.wikipedia.org/wiki/NVLink
[2] http://www.corsair.com/en-us/blog/2014/september/ddr3_vs_ddr... (2channel -> 4channel x2)
GPU's may not be great for every unit of work a RDBMS has to perform, but given their ability to rapidly compute hashes it could help a lot with joins (as evidenced by PGStrom).
And a big on the PowerPC roadmap http://www.nextplatform.com/2016/04/07/ibm-unfolds-power-chi... "With NVLink, multiple GPUs can be linked by 20 GB/sec links (bi-directional at that speed) to each other or to the Power8 processor so they can share data more rapidly than is possible over PCI-Express 3.0 peripheral links. (Those PCI-Express links top out at 16 GB/sec and, unlike NVLink, they cannot be aggregated to boost the bandwidth between two devices.)"
Well yes and no. Both IBM and Oracle have gotten impressive performance using database-specific co-processors, but these are not GPGPUs, they are dedicated hardware that sits on the storage path. Baidu are reinventing that wheel with FPGAs too.
select id, name, age, avg(income) from people group by gender
In this case only the income and gender columns would actually be sent to the gpu and they would do so in a compressed fashion to increase the "effective" bandwidth of data over PCIE. Even more interesting is that id, name, and age, would be pulled from our horizontal store instead of our compressed columnar store in order to minimize the number of iops necessary to fill the result set.
Though, I would like to see Xeon Knights Landing compared to GPUs for db uses.
The weakness of GPU databases is that while they have fantastic internal bandwidth, their network to the rest of the hardware in a server system is over PCIe, which generally isn't going to be as good as what a CPU has and databases tend to be bandwidth bound. This is a real bottleneck and trying to work around it makes the entire software stack clunky.
I once asked the designer of a good GPU database what the "no bullshit" performance numbers were relative to CPU. He told me GPU was about 4x the throughput of CPU, which is very good, but after you added in the extra hardware costs, power costs, engineering complexity etc, the overall economics were about the same to a first approximation. And so he advised me to not waste my time considering GPU architectures for databases outside of some exotic, narrow use cases where the economics still made sense. Which was completely sensible.
For data intensive processing, like databases, you don't want your engine to live on a coprocessor. No matter how attractive the coprocessor, the connectivity to the rest of the system extracts a big enough price that it is rarely worth it.