This comes up in nearly every thread, but NVIDIA’s EULA doesn’t permit using their drivers on “consumer” parts in “datacenters”. So you didn’t include the main reason for the Tesla tax: compliance :).

The “consumer” parts are certainly popular ML workstations, and rightly so.

Yep :)

I'm currently renting LambdaLabs V100 instances at 100% utilization for training https://vo.codes

It's really expensive, and I think I should lean into buying hardware at this point.

I want to build a high end GPU rig, but was wondering how easy the setup was. I've only built "consumer" systems before (2x 1080Ti). Is there any appreciable difference?

Do you have a single card? Multiple? What motherboard do you use?

Do you have any takeaways or resources you can share?

I only have one card. I swapped the Titan V with my 1x1080 in my old, cheap motherboard and it just worked. I had to hook up the water cooling, but I did that because it came with a block, not because I optimized the thermal design. To verify motherboard compatibility, I looked up in the nvidia specs how many PCIe lanes and at what speed I should expect and confirmed in HWinfo that they were active in that configuration -- much like I'd look at a network interface to make sure my 10/5/2.5GbE hadn't turned into 1GbE on account of gremlins in the wires.

I'm not using this for machine learning, so you might want to talk to someone who is before pulling the trigger. In particular, my need for fp64 made the choice of Titan V completely trivial, whereas in ML you might have to engage brain cells to pick a card or make a wait/buy determination.

Probably depends on your budget and if you care about support. Building it yourself isn't a big deal at all, but pretty much every major compute vendor will have a GPU platform as well. That's ignoring nvidia themselves who are the easy button and probably your best bet if you think you'll need support up the software stack:

https://www.nvidia.com/en-us/data-center/dgx-station/

That has more to do with Nvidia policy than cloud but I completely agree. For someone who can just plop 4 Geforce GPUs into their PC it's a much better deal than AWS.

The GeForce makes huge difference here.