Depends on what you mean by throughput. In terms of ram bandwidth on GPU this is very high. PCIE is a different matter. That is still the big limiter on many boxes. Ensuring you have enough cpu pcie lanes to drive all of the graphics cards you want to connect is important. If you are on open power you can use nvlink which is much faster. As far as pulling from external sources we do so using the parquet file format directly or you can ingest csv, json, and xml into our own native file format.

AMD's EPYC offering (or Ryzen) could help fix some of the PCIE bottlenecking. If I remember correctly the EPYC can have 128 lanes, while XEON is limited to 32.

Good Point. I tried to convince someone very recently to go this route so I could piggy back and test on it as well :).