I'm curious: do you use GPUs? It's hard to take this claim seriously. Yes NVIDIA GPUs are expensive, but I don't think we live in the same matrix when you claim that their GPUs are slow and uncompetitive... Unless you have an alternative in mind, in which case I'm all ears.
“ Nvidia GeForce RTX 4060 Ti (8GB) review: Disappointing for $400”
https://www.pcworld.com/article/1925928/nvidia-geforce-rtx-4...
“Do not buy”
“The Nvidia RTX 4060 ti is a waste of sand.”
Wafer prices have increased drastically in the last decade - to the point that midrange cards are no longer midrange priced.
It simply isn't really profitable to make real midrange cards anymore.
In addition, discrete GPUs market is a declining market and has been for 15 years or so. Therefore, in any declining market, the low-end and mid-range products will get squeezed and the high end will get pushed. The remaining buyers of discrete GPUs are willing to pay higher prices. Those who don't will buy a laptop with an Nvidia GPU already in it or some sort of SoC (like Apple Silicon/AMD APUs/Intel APUs).
And if GPUs cost so much to make, why do they get discounted so drastically when the manufacturers eventually decide to compete?
No, this is not true. DIY PCs aren't as popular as before due to the advancements in laptops and interest swinging to mobile. People are buying far more laptops than 10-20 years ago. In fact, gaming laptops outsell gaming desktops 2:1 now.[0] The gap is expected to widen.
The market for discrete GPUs has been declining long before crypto. Crypto just slowed the decline.
Notice the word discrete. The entire GPU market isn't declining. Only discrete.
>And if GPUs cost so much to make, why do they get discounted so drastically when the manufacturers eventually decide to compete?
Part of it is because of price inflation due to covid and crypto bubble. So prices are just going back to a more normal.
[0]https://web.archive.org/web/20210406085311/https://www.idc.c...
Based on desktop CPU sales or based on pre-built sales? I don't know what qualifies as a gaming laptop these days but the best laptops for software development often have discrete graphics cards in them e.g. 1660 Ti
I'm guessing it's having a non-APU AMD/Nvidia GPU inside the laptop.
But it makes sense to me. Laptops have gotten much better, and most gamers aren't buying Nvidia 4090. The most common GPU is just a GeForce GTX 1650 which many laptops have and have thermals that easily fit inside a laptop.
It's not surprising at all that gaming laptops outsell gaming desktops. This isn't the 2000s anymore where if you want to play PC games, you build a DIY desktop tower and buy a discrete GPU.
It's a myth that most PC gamers use top of the range GPUs. Most of them use low-end or mid-range GPUs.
This doesn't show what you think it does. The majority of desktop gaming PCs are self built and won't show up in that statistic.
Additionally GPUs are the most commonly upgraded component during the lifetime of a built.
It probably does. I never paid for the report but I assume IDC collects data on DIY desktop towers and make an educated guess.
Regardless, gaming laptops sells far more than DIY computers.
[1] Second graphic: https://www.tomshardware.com/news/nvidia-maintains-lead-as-s...
And it's not clear what % of them were used for gaming vs crypto mining.
The A100 which is similar to 4090 in silicon goes for 15-20x. Nvidia is probably deriving as much profit from cloud at this point as gamers.
(admittedly I have zero interest in them as gaming hardware, but on the GPU compute side this is definitely the case)
The low-end is crawling along due to fixed cost overheads from PHY area that doesn't shrink. 4N is ~3x the price per area as Samsung, and PHY area becomes relatively much larger. The incentive is to cut every cost in PHY area - fewer memory channels to reclaim PHY area and using cache instead, cutting PCIe width to reclaim that PHY area, even cannibalizing the media engine/encoders to claw back that last little bit of space. AMD has actively been engaged in this battle with 6600XT (x8 PCIe) and 6500XT (x4 PCIe, no encoder) and RDNA2 generally shrinking memory bus and replacing it with cache in this same way (and 6600/6600XT and 5700/5700XT also regressed performance vs their predecessors in some situations just like 4060/4060 Ti). On top of that you have big fixed increases in manufacturing, shipping, etc, and the rest of the BOM is ballooning over time too (forget VRAM spot prices, ask automakers if their BOM is higher or lower than 2019... and it's not a small difference, it's probably 2x!)
In this world of organically-low performance increase within the low-end product segment, the impact of clearance sales exceeds the impact of new product generations. And this means we end up in this situation where reviewers do the "the new products suck, buy the ones we told you sucked last year" rubber-chicken routine year after year after year. But if everything is constantly bad, kinda nothing is, really. That's just the nature of the product segment.
And when you look at things like the 3060 Ti being $275 on clearance... the market consensus is that the old products are Good Enough. And the new products in fact may be worse in some ways, unless you are willing to step up in price to maintain the same product segment rather than the same price segment. Because TSMC is cranking the prices 25-50% every generation and Samsung was abnormally cheap to begin with, so there is a definite price step happening.
Hard to see how people don't understand that the $200-300 product segment is dying in the same way the $100-150 product segment already died. 6 months after launch you could buy a 7850 2GB for $150, or a 7750 for $100. What does $100 buy you in the new/retail GPU market these days? Does anybody actually seriously think the inability to deliver new $100-150 GPUs with enthusiast-tier performance is due to "greed" or "agreeing not to compete" as opposed to market realities? But people will defend it to the death that the privilege of making $200 GPUs is some massive profit opportunity that AMD and NVIDIA are just choosing to ignore.
We are in the end of silicon in some ways. Post-Moore's Law the costs have been spiraling, and now we are to the point where consumers are deciding that no, it's not worth the cost increases. And companies aren't going to cut their throats and run zero margin or sell at a loss either. And that eventually flows through to TSMC and ASML, and node research will slow (hyper-NA is already effectively canceled) and silicon research becomes incremental and iterative and slower rather than continuing to make even N7->N5->N3 sized progress. It can get slower.
Some people just have an ax to grind and will contort their facts to agree with preexisting thoughts and feelings. I think it's called cognitive dissonance.
NVIDIA is definitely not 'competitive' in the sense of participating in a competitive market, or in the sense of being characterized by furious striving for rapid and continuous improvement. Many of their newer cards are barely 'competitive' with their own cards from a generation or two ago.
It's extremely clear to me what people mean when they say that NVIDIA or their GPUs are not meaningfully 'competitive'. I don't think it's that hard to see, actually.
The 4060 is the only card this generation that's a flop, and I wouldn't consider one in four being "many".
> I don't think it's that hard to see, actually.
It is if you're not looking or trying to be disingenuous. Every generation has introduced new features and had noticeable impacts on the consumer products in its category (games). DLSS alone has been a game-changer, pun intended.
https://arstechnica.com/gadgets/2023/05/review-amds-269-rx-7...
“AMD fails again: RX 7600 review”
That’s… very debatable. Don’t get me wrong, they’re definitely competitive at all levels but AMD absolutely is too & very arguably is better in some cases.
Plus while cards like the Nvidia RTX 4090 rank very highly they cost 60+% more than the extremely comparable AMD RX 7900 XTX.
That’s definitely not “best at any price”.
That's the PC consumer market. Every Xbox and PlayStation shipped in the past decade has had an AMD GPU.
NVIDIA had $11bln in 2022 revenues from graphics vs. AMD's $6bln.
Two; iOS has a minority share of the market too. Less than a third of it. Guess you’d also call Apple unsuccessful in the mobile space?
No. No you would not. So, clearly market share of one segment is not the be-all end-all you are claiming.
The RDNA line is targeted at gamers. The reasoning apparently being "let's be careful that these RDNA GPUs are bad enough so they can't really be used for anything else but games; if the user is into compute, they should pay the big money for CDNA GPUs".
This is also reflected into the ROCm situation. ROCm has good support for CDNA GPUs, which is the hardware that is deployed in the huge national-lab GPU projects, where money is not an issue. The problem with that approach? normal people do not have access to CDNA GPUs that cost a multiple of 10000 USD apiece.
Normal gaming nVidia GPUs fill that role. You need big guns, you buy a lot of their compute cards next.
AMD had decent compute capabiliites in their CDNA line, but they didn't follow through with drivers. The cards were pretty bad to try to use for compute with early ROCm, and just as bad for gaming.
I think he meant Nvidia GPUs are more expensive than Nvidia GPUs from previous generations.
In what way has Nvidia abandoned the consumer market? They have something like 80%+ market share on discrete gaming GPUs and still rake in billions a year from GeForce, which at their revenue is a sizable fraction.
And what makes you say that current AI hype is a grift? Do you truly think that the work with transformers is just a fad? I swear people on this site were saying the same thing about AI in general 3+ years ago before transformers blew up the scene.
We’ve been here before. ChatGPT isn’t functionally all that different from ELIZA from the 1960s. It’s tricking people just the same, into thinking it’s more than it really is.
I’m also eagerly awaiting the discovery process that will show that all of this generative AI was built on mass copyright infringement. OpenAI certainly ingested the entire z-library. They’d have to be stupid not to, right? If they didn’t use it, less ethical competitors would have beaten them to market.
Oh, I’m sure they used some sketchy subcontractor to do it. We’ve all seen a variation of this in our careers, where a dataset was acquired under questionable circumstances, and you don’t ask and don’t tell if you want to keep your job, right?
And that’s just speculation. There’s hard evidence for the Getty Images lawsuit.
Regarding the questionable training data I agree, but the likely outcome is that those AI models will be used regardless.
They kind of weren’t super wrong. What people usually mean by this is that the product doesn’t match the hype. It very much didn’t.
Even now we’re only starting to catch up to the breathless promises from back then. Barely.
The main difference now is that the barrier to entry is lower so it’s starting to get some interesting uses cases come out
One of it's only recurring warnings was the danger of underestimating what a "stochastic parrot" can do in practical terms.
I expect people will go nuts about it again in a few years before temporarily abandoning it.
At least that was the case in 2011, 2013, 2017 and 2020-21.
I don't see any value in crypto projects personally, but if you think it's going away forever, expect disappointment.
Pretty much everyone has heard of crypto by now. The Ponzi scheme is out of marks. There will be no next time. Sell now if you’re still holding the bag.
Combine that with the stuff that Mythic was working on in texas (before they ran out of burn) and we're looking at a world where, most people won't have a need for heavy GPU compute.
And I’m not even touching the fact that FPGAs (which I love to work with!) are a pain to program and optimize for.
It’s not that you can’t use them, and there may be narrow cases where they have a benefit. There are always trade-offs to be made.
But that doesn’t make them the best hardware for AI (there’s too much overhead) and it doesn’t change the fact that they’re much harder and slower than bare metal CUDA to optimize for, in a fast moving field.
Look at it this way: if you have a fixed DL architecture and a fixed set of network weights, there’s no question that you could come up with an ASIC that only does that, that will be cheaper than an FPGA or GPU, faster, and use lower power too. It will be the perfect inference solution for that specific problem.
It would also be obsolete in a few months, before the chip comes back from the fab.
That means an ASIC.
They have both been at it gradually increasing prices to the point where a mid range card of a decade ago now costs 5x as much and twice the price of an entire console. It's got pretty insane this generation with marginal gains and massive price hikes and another set of renaming cards to step the dies down again. Finally no one is buying them but these companies are big enough they will just blame the overall economy anyway.
Is has been the years of investing and specially the software ( CUDA ) that keeps Nvidia as the "king". Hardware wise AMD it's pretty much up there or winning in a few cases.
AMD has the patents for putting a SSD directly on the card, they could be killing it in the home AI market, but.... they just can't get it together.
Radeon SSG was just a PCIe switch chip on-card, with the SSD+GPU behind it, so functionally it is the same as having the SSD on the motherboard.
I'm not sure if they got patents but either way the problem it solved was not the one people think it solved. It wasn't "using the SSD as memory", and in the way that it did (block storage) any other SSD+GPU can function identically in the same way without needing the SSG tech. Putting the SSD on your mobo performs as high in every way on any GPU (CUDA RDMA Direct have been around for a while, since Fermi/Kepler at least).
It was just a convenience thing of getting a SSD mountpoint built into your GPU basically. They even showed up as a HBA in windows and were treated as a striped disk.
https://youtu.be/-fEjoJO4lEM?t=62
More recently this idea has re-surfaced with some of the modern GPUs with x8 interfaces (6600XT, 4060 Ti, etc) getting a couple M.2 drives in the other x4x4 via bifurcation, and this functionally performs the same as SSG but without the switch chip (needs hardware support instead). And if there are patents around using a switch chip that may be how the patent is evaded. But combination cards utilizing more than one type of hardware in general is not that novel (network card/ssd combos are another popular one) and I'm not sure AMD patented it or it's defensibly novel against prior art.
https://www.tomshardware.com/news/asus-demos-geforce-rtx-406...
Investing in AMD at $9/share is one of my best investments.
Hard disagree as someone who's had two 7900 XTXs and is now looking to buy an Nvidia card. If I need to pay 1100€ for a product, I'd like it to work and AMD cannot deliver on that.
The lack of a recall for broken hardware also makes me distrust them from a customer support point of view.
Have you ever heard of the Geforce 8 Series "Bumpgate"?
Bumpgate was an industry-wide problem and pretty much nobody recalled for it unless they were sued into doing it.
It's really just Nvidia firmly winning the entire market the good old fashioned way. Those who can compete still can't hold a candle, and the market at large is content to keep buying Nvidia because their stuff is simply That Fucking Good(tm).
It's Nvidia's market to lose, and they clearly aren't losing any time soon.
NVIDIA has not had a problem like 5700XT or Vega drivers in the modern era, and they haven't even had a more short-term shitstorm like RDNA3 launch drivers that lingered for nearly as long. And there have been multiple instances of top-5 e-sports titles being flatly broken on AMD drivers (often resolveable by going back to much older drivers) for prolonged periods of time (quarters/years) that simply don't happen on NVIDIA cards.
But of course there is a low-level stew of problems on both brands constantly. Power-saving with multimonitor is a great example of one that's been perma-broken for 10 years on both brands now. But there is also a high-level stew of Radeon-specific problems that pretty constantly churns and it gets discarded out of hand because "I've never had a problem" like that means 5700XT didn't exist.
even working from a baseline assumption of intellectual honesty it's a super frustrating discourse overall, and I think in many cases the assumption may be unwarranted.
You actually need to do more. The ML community chose nVidia because a decade ago, nVidia donated GPUs to universities for free. People like Hinton hacked CUDA as a poor man's HPC.
For example the original AlexNet use 224x224 resolution pictures, to fit the GTX 580 3GB memory.
AMD definitely wants to compete. Why wouldn't they want a piece of a trillion dollar market?
The problem is that AMD was almost bankrupt, had poor leadership until Lisa Su, and they were just trying to survive making console chips for Sony and Microsoft until Zen2.
The MI 300 and MI 250 are making huge inroads to the point where they are finally pulling folks away from CUDA.
How much does one MI300 cost, BTW? Is there a price published? are they available to buy anywhere?
huh :o) what are your thoughts about intel on this ? it was the market leader in cpu's for ages, and decided to push fpga's for machine learning !
in the hindsight, it seems like a serious misstep.
It's true that modern GPUs are ridiculously expensive compared to where they were 10 years ago, but there still is a good sub-$200 GPU available: the RX 6600, from AMD. It's not the fastest, but from what I understand, it gives the most power for your buck of any modern GPU.
> But instead AMD has just followed Nvidia into making slow, uncompetitive GPUs at high prices.
I am not really sure they could do that. If nvidia cards are cheaper to produce, then for AMD just ordering more cards can end in massive losses short term, and much less profit long term, as customers demand cheaper cards. Nvidia always could lower their prices as the updog, and hurting AMD really bad.
But yes, in retrospective, knowing that the demand for GPUs would remain ridiculously high for years, the best move for AMD was ordering and producing much much more cards. As the best move for us customers were jumping into crypto. (Not necessarily the best strategy.)
This seems like a needlessly conspiratorial and complicated cope to avoid recognizing that the low end is simply suffering from fixed overhead and rising costs.
The alternative/null hypothesis is that both companies are responding to the market options that TSMC and BOM costs provide them, and the technology is simply moving slower in some segments. In this null hypothesis, it's not an active conspiracy to screw anyone, and nobody is deliberately "followed anyone into noncompetitiveness", there simply isn't a big market opportunity to sell $150 enthusiast GPUs anymore (7850 HD!) and make a decent profit based on the actual costs of building the product and getting the product through design/validation and manufacturing/shipping cycles.
Like, it's kinda facially absurd that gamers think that there's some golden opportunity to make low-margin $150 GPUs that everyone is just choosing to ignore "to push margins". That's a shitty low-margin product and everyone is choosing to ignore it because it's not profitable, gamers are just kinda operating under this fallacy that $150 is a lot of money for an enthusiast-tier product.
Competition means giving customers exciting, fast, low cost GPUs.
Nvidia, as #1, no longer needs to compete and has stopped winning via low cost, high performance GPUs. This opened a giant opportunity for AMD GPUs to give those things to consumers and start winning against Nvidia.
But instead AMD has just followed Nvidia into making slow, uncompetitive GPUs at high prices.
So in a real way AMD is keeping Nvidia at #1.