No gamers need such high VRAM, if you're buying Gaming cards for ML work you're doing it wrong.

It's Nvidia that considers them, "gaming cards". The market decides their use in reality though.

Their strategy is to sell lower-VRAM cards to consumers with the understanding that they can make more money on their more expensive cards for professionals/business. By doing this, though they're creating a gap in the market that their competitors could fill (in theory).

Of course, this assumes their competitors have half a brain cell (I'm looking at YOU, Intel! For fuck's sake give us a 64GB ARC card already!).

And if you buy the cards that Nvidia says are for gaming and then complain that they don't have good specs for ML, who is the fool exactly?

Games already exceed 16 GBs at 4k from years.

I exceed 16GB in Chrome.

That says more about Chrome than anything else.

I use Firefox and have an 8Gb card and only encounter problems when I have more than about 125 windows with about 10-20 tabs each.

Yes, I am a tab hoarder.

And yes, I am going to buy a 16Gb card soon. :P

1200 tabs isn't that many

That's why I need to get a 16Gb card. :P

System Ram != GPU VRAM

> Gaming cards for ML work you're doing it wrong

lol okay. "doing it wrong" for a tenth of the cost.

And screwing gamers over by raising the prices by 2x. Fuck that.

Believe it or not, it's possible to be interested in both machine learning and videogames. That's ignoring the notion that it's somehow how screwing over gamers. Buy a fucking AMD card. They're great at gaming and you don't need CUDA anyways. Enjoy the long-term acceleration of GPU performance increases you're getting by the way. All that stuff comes from innovations made for workstation/DL setups.

Get an AMD GPU? Said no one ever.

It seems like the 90-series cards are going to be targeting prosumers again. People who play games but may use their desktop for work as well. Some people are doing AI training on some multiple of 3090/4090 today but historically the Titan cards that preceded the 90s cards were used by game developers, video editors and other content developers. I think NVIDIA is going to try to move the AI folks onto Digits and return the 90-series back to its roots but also add in some GenAI workloads.

forget the post but some dude had a startup piping his 3090 to use via cloudflare tunnels for his ai saas making 5 figures a month off of his 1k gpu that handled the work load, I'd say he was doing it more then right.

And if his volume grows 100x should we expect him to run his company off gaming gpus? Just because you can do something doesn't mean you should or that it's ideal.

There's a reason large companies are buying H100s and not 4090s. Despite what you guys think, serious ML work isn't done on the consumer cards for many reasons: FP16/FP8 TFLOPS, NVLINK, power consumption, physical space, etc.

MS Flight Simulator 2024 can consume...who knows how much.

I know my 10 GB 3080 ran out of VRAM playing it on Ultra, and i was getting as low as 2 fps because I'm bottlenecked by the PCI-Express bus as it has to constantly page the entire working set of textures and models in and out.

I'm getting a 5090 for that, plus I want to play around with 7B parameter LLMs and don't want to quantize below 8 bits if I can help it.

I've regularly exceeded 24 GiB of VRAM in Microsoft Flight Simulator 2024. Imagine a huge airport environment with high levels of detail, plus AI aircraft in the ground and sky. Then, on top of that, terrain and textures of the surrounding environment.

And that's at 1440p, not even 4K. The resulting stutters are... not pretty.

tell us how to do it right.

Get your daddy's credit card and buy H100s like a serious person.

If the VRAM wasn't small, the cards would all get routed to non gaming uses. Remember the state of the market when the 3000 series was new?

Then they should sell more of them.

They can only make so many, that's part of the problem

They should contact Intel.

Why sell more when you can sell less for more

Maybe, but if they strapped these with 64gb+ wouldn’t that be wasted on folks buying it for its intended purpose? Gaming. Though the “intended use” is changing and has been for a bit now.

The only reason gaming doesn't use all the VRAM is because typically GPUs don't have all the VRAM. If they did then games would somehow find a way to use it.

Game engines are optimized for lowest common denominator, being in this case consoles. PC games are rarely exclusivities, so same engine has to make it running with least ram available and differences between versions are normally small.

One normally uses some ultra texture pack to utilize current gen card's memory fully on many games.

Consoles would have more VRAM too if these cards had more VRAM. It's not like they're made separately in isolation.

XX90 is only half a gaming card it's also the one the entire creative professional 3D CGI, AI, game dev industry runs on.

Not really, the more textures you can put into memory the faster they can do their thing.

PC gamers would say that a modern mid-range card (1440p card) should really have 16GB of vram. So a 5060 or even a 5070 with less than that amount is kind of silly.

hmmm, maybe they can had different offerings like 16GB, 32GB, 64GB, etc. Maybe we can even have 4 wheels on a car.

So you're saying more VRAM costs more money? What a novel idea!

Conversely, this means you can pay less if you need less.

Seems like a win all around.

Saw someone else point out that potentially the culprit here isn’t nvidia but memory makers. It’s still 2gb per chip and has been since forever

GDDR7 apparently has the capability of 3gb per chip. As it becomes more available their could be more VRAM configurations. Some speculate maybe an RTX 5080 Super 24gb release next year. Wishful thinking perhaps.

Totally agree. I call this the "Apple Model". Just like the Apple Mac base configurations with skimpy RAM and Drive capacities to make the price look "reasonable". However, just like Apple, NVIDIA does make really good hardware.

Makes sense. The games industry doesn't want another crypto mining-style GPU shortage.

Is there actually less VRAM on the cards or is it just disabled?

GPU manufacturers have no reason to include additional memory chips of no use on a card.

This isn't like a cutdown die, which is a single piece with disabled functionality...the memory chips are all independent (expensive) pieces soldered on board (the black squares surrounding the GPU core):

https://cdn.mos.cms.futurecdn.net/vLHed8sBw8dX2BKs5QsdJ5-120...

Well, they are gaming cards. 32GB is plenty for that.