NVLink has 80GB/s [1]. DDR4 quad channel (Xeon Servers) has ~120GB/s [2]. So no this rationalization doesn't fall apart. Furthermore in the event NVLink gets faster then RAM, then you'll still be bottlenecked by RAM access, as you'll buffer here.
This of course is ignoring weird systems where you attempt to maintain ACID coherence of tables between GPU, CPU, and Disk memory. But then GPU memory size become inherently limiting as even the biggest max out at ~32GB.
[1] https://en.wikipedia.org/wiki/NVLink
[2] http://www.corsair.com/en-us/blog/2014/september/ddr3_vs_ddr... (2channel -> 4channel x2)
GPU's may not be great for every unit of work a RDBMS has to perform, but given their ability to rapidly compute hashes it could help a lot with joins (as evidenced by PGStrom).
And a big on the PowerPC roadmap http://www.nextplatform.com/2016/04/07/ibm-unfolds-power-chi... "With NVLink, multiple GPUs can be linked by 20 GB/sec links (bi-directional at that speed) to each other or to the Power8 processor so they can share data more rapidly than is possible over PCI-Express 3.0 peripheral links. (Those PCI-Express links top out at 16 GB/sec and, unlike NVLink, they cannot be aggregated to boost the bandwidth between two devices.)"
Outside of CPU >> GPU, I'm not sure what other data movement you could be talking about. A SAS HBA or Ethernet NIC or Infiniband HBA are almost always going to be operating over the same PCIe bus the GPU uses. In the rare instances they're built onto the CPU, the "network link" is likely going to still be significantly slower than the fastest PCIe slot.