Not in SLI, no. SLI is a technology to have two GPUs act as one, by syncing among themselves. When you have two GPUs in SLI, your CPU program will generally see them as a single GPU with (almost) twice as many cores and twice as much bandwidth, but the same amount of memory.
For this to work, the GPUs need to be very similar (i.e. you can't SLI a TitanX and a 1080).
You can, however, put many[1] different GPUs in the same system, and then the onus is on the CPU program to properly manage and send work to each one separately. This is not SLI.
Generally, SLI is useful for games where the program assumes "a GPU" and sends work to it, and you want to have more performance than your single card can provide. For compute/ML, you're much better off just using each card directly.
[1]: Don't quote me on this, but I believe up to 8x is supported, above that is uncharted territory.
- Is it possible, officially supported way, to have both GTX and Quadro at the same time in the machine? I prefer GTX for rendering speed (Redshift renderer), but I need 10-bit output Quadro has which GTX doesn't have.
- Are there GTX (1080 TI in this case) cards out there which don't take double PCI space? I don't need multiple outputs and I can hook them up to the water block, if needed. I need as many cards in a single workstation case as possible for rendering. I've seen people saw/solder off stuff from cards in order to get at that, but not something I would do personally. Ideally I would prefer to have eight 1080 TIs in a single machine along with a single (modest) Quadro for 10-bit output to three monitors - are there even motherboards that could support that?
Note that I'm looking for workstation / deskside solution, not server 747-fan-sound-behemoths.
I would like 12-bit output, but that's not possible due to limitations in X11.
Interesting that you mention this as I've been searching for an answer before I build a ML rig and embark on learning it. Is there any benefit for ML performance if I put a GTX 970 and a 1080[ti|x] in the same machine? Just trying to figure out how to re-use spare parts.
Both have max 32 PCI Express lanes - so you can theoretically hang max 2 PCIx x16 cards off each CPU, but as few of the lanes is reserved for other peripherals, in 4GPU/CPU setup they will practically run at lower speeds, ie 8/8/4/4. However, using system with PCIe root complex [1] can improve the GPU<->GPU communication speed.
[1] https://www.microway.com/hpc-tech-tips/pci-express-root-comp...
so GPUs get better bandwidth (P2P etc).
NVIDIA said the 10-series can only do dual-SLI at most (two cards), nothing more.
In cases where you want faster cross-card transfers and are willing to mess with HW configurations, there's NVLINK.
Wiki: https://en.wikipedia.org/wiki/NVLink
PR piece: https://blogs.nvidia.com/blog/2014/11/14/what-is-nvlink/