Steve from GamersNexus has a good video about this. His argument is similar to yours -- nVidia is marketing this card as a god-tier gaming card (and is of course very powerful for this purpose) but practically it's better suited for people who need high VRAM + strong compute on ML/productivity tasks. His argument being that, in terms of price to value of 3080 vs 3090 for regular ol' gaming, the 3080 is a better choice, and really access if you actually need something as intense as a 3090.

Except the 3090 doesn't get the Titan driver, so the 3090 is still artificially slow at a variety of productivity workloads (see Linus Tech Tips review where they checked a few of these).

Aka, it's a more expensive 2080 Ti, not a cheaper RTX Titan.

How can the 3090 be meant for ML when it has intentionally gimped ML functionality?

In the end, the RTX 3090 is still better at ML workloads than every card other than the RTX A6000 or the A100. Although the 3090 theoretically falls behind the V100, Titan V and Titan RTX in terms of pure mixed-precision matrix math, in practice the 3090 still performs better on almost all workloads.

So maybe not 100% designed for ML, but better than anything else out there unless you want to sell a kidney.

Because in reality, it’s only gimped for certain scenarios, and compared to the 2080TI, it’s a monster upgrade for ML!

Here’s a dirty secret: there is a ton of ML prototyping done on GeForce level cards, and not just by enthusiasts or at scrappy startups. You’ll find GeForce level cards used for ML development in workstations at Fortune 50 companies. NVIDIA would love everyone to be using A100s to do their ML work (and V100s before that,) but the market isn’t in sync with that wish. The 2080TI remains an incredibly popular card for ML even with only 11GB. Upping to 24GB, even with the artificial performance limitations for certain use-cases, enables new development opportunities and use-cases to explore.

When it comes to product stratification, the hard rule according to driver EULAs is that GeForce cards can’t be used in data centers. For serious ML development at scale, NVIDIA has their DGX lineup. In the middle are the Quadro cards, but they tend to be a poor value for ML. The cost differential with Quadro is largely due to optimizations and driver certification for use with tools like Catia or Creo (CAD/CAM use,) which don’t intersect with ML.

The Titan RTX may not have the gimped drivers, but the 3090 beats the Titan in many benchmarks nonetheless. Is the 3090 the best NVIDIA PCIe form factor card for ML? No. The A100 is still king of the crop and is the only Ampere card with HBM memory, and even the A6000 will outperform for many use-cases with 48GB of RAM. Still, the 3090 will be the optimal card for many.

I’m one of the lucky few to have a 3090 in my rig. I lead of team of volunteers doing critical AI prototyping and POC work in an industry give-back initiative, and price was not a leading factor in my decision to procure a 3090 over a Quadro. I chose the 3090 principally because I didn’t want a loud blower card in my computer (and I don’t need 48GB.) If someone donated an A100 to our efforts, I’d gladly take it, but it wouldn’t replace the 3090. It’s not a graphics card and it won’t play games, which indeed is an important value-added benefit of the 3090 :)

> gimped ML functionality

curious what is that?

The 3090 supports but doesn't let you enable the titan driver paths, SR-IOV, NGX, and critically, is capped to half-rate tensor operations with FP32 accumulate.

An official firmware release could change this but they're likely saving it for the Titan cards down the road.

Where do you see that this is artificial (i.e. in software/firmware) rather than just missing some hardware that's present on the Titan/Quadro cards?

The same thing happened with the 2080 / Titan and looks to be confirmed for the 3000 series by AnandTech. https://twitter.com/RyanSmithAT/status/1301996479448457216

The Reddit AMA implies the Ampere architecture supports this configuration but is software limited under the FP32 section: https://www.reddit.com/r/nvidia/comments/ilhao8/nvidia_rtx_3...

To epistemologically correct, I don't believe one will ever see a statement from NVIDIA confirming that this is a software limitation. I believe it's just inferred.

edit: This is close to confirmation: https://imgur.com/a/RH8vyz9 -though I may suspect there are non reversible hardware fuses at play too.

They also did this with the TITAN in the past, I think, so it's not entirely unprecedented.

TITAN X got access to pro drivers shortly after AMD announced the Vega Frontier Edition.

Maybe it is a type of binning. Maybe the machine learning specific hardware is more prone to errors so they just disable that on subpar chips and sell them for use in graphics cards?

The AMA section on FP32 seems to imply it's in hardware though, unless I'm missing something – datapath is a hardware term, at least.

I mean, I really hope that it's just in the driver so that an enterprising reverse engineer can hack the driver and re-enable full FP16/FP32 accumulate :)

Wouldn't the Titan series be more meant for ML?

Let's be real, the only reason the 3090 isn't a Titan card is because Nvidia saw this coming and didn't want AMD to be able to top their "untouchable" Titan brand.

>3090 is actually a Titan without the Titan name.

AdoredTV made an entire video[0] to demonstrate this is absolutely baseless conjecture that NVIDIA Marketing tricked people into inferring.

It absolutely destroys the Titan idea, leaving no room for doubt.

[0]: https://www.youtube.com/watch?v=s23GvbQfyLA

I've gotta watch 30 minutes from some tech tuber to get your point? C'mon dude.

I was curious so I watched. It's a good video. I ended up going with the 3090 because of the extra vram and I'm familiar with CUDA but I'm not under the impression that this thing is a "Titan" even if it is all arbitrary corporate market segmentation. Relevant bit is at 15 minutes: https://www.youtube.com/watch?v=s23GvbQfyLA&15m1s

People who hate video can process a 4 page article in 15 seconds rather than minutes

There is no reason AMD wouldn't pull their own "Titan"-class GPU. A 6900XT with 32GB of HBM would destroy anything NVidia could muster.

I mean they kinda already did put out a Titan class card. My point was that the 3090 is a Titan in all but name, and AMD unveiled their very capable response today.

I'm saying the only reason Nvidia didn't call it a Titan was because they knew it wasn't going to have an unassailable advantage over AMD.

There's no Titan-"class"

A Nvidia card is Titan if it has the name and the drivers, and not if it doesn't.

You're no fun.

AMD Titan-class cards tend to have HBM. I'm saying that AMD can very probably still do even better.

AMD has a CDNA in their pipeline which they haven't talked about in a while. That's their compute oriented architecture, whereas RDNA is graphics-oriented.

Indeed. I still think it might be possible to see an RDNA2 HBM card.

I thought Quadros were meant for non-gaming workloads and the Titans were just more powerful than the top-tier mainstream/gaming card? Or have things changed?

Yes, Quadro was for non-gaming workloads. Titan meant a different thing with every new iteration. Some were explicitly "non-gaming card that you can play games on" other were "got tier gaming card".

RTX Titan for example was not as good in FP64 calculation compared to RTX Quadro. While first Titan was good at both. Nvidia kept changing what Titan means in order to extract most amount of money without hurting Quadro sales.

Even at its peak certain things were software limited for Titan, but not for Quadro. Nvidia has tons of artificial limitations for each of their graphics card once you go beyond gaming.

There's no titan this time around and nvidia have been talking about simplifying their lineup and implying the 3090 is to fill the titan shaped hole in the lineup, but also leaving them room to release a RTX Titan Ti or whatever if they change their mind. Also I imagine to head off consumers who would otherwise be like "Maybe I'll wait for a 3080 Ti".

I also expect been a kind of struggle between their consumer side who need some other reason to sell their top end GPUs beyond just the whales when game developers aren't really that interested in pushing out features that will require it given the niche ownership, while the datacenter people want to protect their Quadro margins. This to me seems their most likely reason for the on-again/off-again relationship with pushing their high end cards for compute usage vs gaming usage.