(which is the more likely scenario because they can make much more money on enterprise/datacenter products, see DRAM)

Testing hardware is a complicated matter (you can do a PhD on the topic), and nvidia probably does a better job at it than a hacker in their basement who thinks they can upgrade their GPU for free.

On the other hand ... perhaps a few bit errors is not a problem for AI workloads ... but they could also crash your system ...

Except he literally did.

Which is why I think these devices are more likely to be faulty.

I don't know what Intel is doing as I quit keeping up with them. I do miss the core unlocked boards that you'd put a 2 core cpu and maybe win the lottery on and get all 4 cores.

I recently had a faulty A100, for some reason the provider turned off ECC on that card, so it silently failed and gave junk results... Took me a minute to figure out.

Anyways, that's what I tell myself as I taste my bowl of sour grapes.

That said, 80G for something that you picked up for cheap could be an absolute steal these days.

Currently listening to the soothing drone of my $200 Octominer cooling my cards while I'm fine tuning a model (almost makes a good space heater).

https://www.reddit.com/r/LocalLLaMA/comments/1wjr59t/768gb_v...

How are you dealing with the RAM limit (32GB)? I had upgraded mine to an i7 but it was unstable so I’m back to the stock Celeron class CPU.

Bad RAM isn't a huge issue as you can flag and ignore the relevant regions and be down a few MB.

I even looked into blog posts of using them or unlocking them somehow but I never thought to think that they actually had the full HBM memory and all just fused away??? wow

After which I sold most of them for $2000-$2600 (I believed the panic about the new mac studio launch) instead of holding them to sell for like $5k and making my annual income in a week. And also I failed to do the capacitor mod after buying a couple hundred in soldering equipment. But at least I sold most of them for monero, and that's gone up since and allowed me to avoid massive ebay fees.

The moral of the story here is that you can fantisize about getting in at the bottom, but even if you did you may not have rode it to the top. Plenty of people bought bitcoin when it was half a cent and sold at the local maxima to net $30k and spent it on a used corrolla.

I think my story may make your grapes less sour.

Edit: I knew at the time of purchase that the 10GB->80GB unlocks were problematic but the 8GB->64GB cards were solid. The pricing reflects this, so thats just skill issue.

As for v100, p100 etc I considered them, but they make zero sense as they don't have int8/int4. Everything gets upscaled in them which slows the inference down.

Hard to believe $1k was a deal a few months ago (and $200 before then).

The highs and lows of my predictive capabilities

Annoying thing is finding a good motherboard to put multiple blower cards in. Mining rigs are a steal but PCIe x1 limitation is annoying. Got a bunch of Volta cards too sitting around until I figure out where to put them (and the performance per watt sucks next to the 170).

That's why this works. The yield on those HBM stacks is excellent. The binned cards had failures in the logic die, and hyperscalers want uniformity in their clusters. That's why the mining market took the downbinned cards.

Mounting the HBM on the interposer has to happen before putting the interposer in the package.

So it's the "must be in-package" yield tests that drive this.

It is much more of "let's sell cards to crypto people, but nerf them, to not harm our enterprise margins", not "we've got a ton of subpar hardware, let's to something with it".

wtf, it seems RAM truly was too cheap back then...

Plus: most of those cards probably have broken RAM chips anyway. Throwing them out would be a waste, limiting trash chips to a small amount of RAM that does work turns recycling costs into a product that can be sold for money.

In addition, fabricating the same die for every SKU point in the product line means the maker does not have to guess how many version X vs. version Y vs. version Z dies to fabricate. The just have one fabricated. Then the dies become SKU version X, Y or Z as demand requires during the card assembly process.

Wait, so does this unlock it to actually do something useful, or just "mine" marginally more shas?

So I guess the next step is to stop selling the lower-binned card? https://www.tomshardware.com/pc-components/gpus/nvidia-repor...

0: https://news.ycombinator.com/item?id=50452839

It’s also useful for binning: These lower spec locked products can have defects in the unused parts, like flakey memory regions or channels. Not every CMP170 can be fully usable after unlocks.

The alternative to this market segmentation isn’t “Everyone gets the whole thing for the lowest price”. It’s that the lower price version doesn’t exist at all, or if it does it’s a lot more expensive.

Basically the high price of the fully unlocked models is what makes it possible to sell artificially limited models at the lower price point (while also offering the ability to possibly sell lower binned units). Basically you can gas the profit on the high-end to drive down the cost on the low end, but obviously this only works if you have a distinct High and Low end.

People think that if we banned this practice it would lead to the top shelf models being sold at the bottom shelf price. But that's just naive market confusion. A ban would just mean that the top shelf unit at a higher price and the lower shelf unit at a higher price would be available.

There is no solve to the "problem" that results in "High shelf unit being sold at lower cost", in fact the thing you are complaining about is already the closest to that ideal, as the manufacturer can leverage economies of scale to drive down costs across the product line.

At the beginning of a product, when yields are bad, I could get a low tier chip at a good price, but then as yields get better, low tier chips wouldn't be available and I no longer have a low cost option. Presumably higher yields would lead to reduced prices on the higher tier chips, but I don't think they'd get down to where the low tier chips were.

Another alternative is that the maker only ever sells chips that test successfully as high tier, but I think it's repugnant to not make use of parts with defects if the defects can be gated off.

Each wafer is printed separately. They don't build new custom production line for each new design.

That said, with Zen6 having 12 cores per chiplet, I wouldn't be surprised if they eventually end up designing a smaller chiplet with 6 cores and a dedicated small chiplet carrier. Or push 6 core SKUs to the APU side that doesn't use chiplets and I think has small core count variants.

All sorts of reasonable interactions are then policed by people with visceral reactions to normal activity.

That one price would be somewhere between the low price and the high price. So, the world you want to live in would be one where "the poor" who currently pay the low price would need to pay more. And "the rich" would get a discount from their high price they pay currently. That seems morally repugnant beyond words.

At least not the development of GPUs.

Klein-bottle-shaped leopard's eating face

Buy a shitton of these things, spend a year working on the unlock. If you find one, sell the shitton at 5x higher price (which is what happened).

I disagree: If there's any risk that the fuse will blow with normal usage, then a blown fuse means a defective product.

Joke aside, good investigation and blog debut, kate!

I just don’t know if I’m brave enough (and knowledgeable enough) to go this blown fuses way. Seems a lot more risky.

But I can't see any independent confirmation and having personally tried to contacted the paper author I received no response. So, I'm not sure the paper findings actually hold.

I'm now on the process of investigating this and, if I achieve any conclusive result, either for or against, submit a new paper on the subject.

Oh wait, cars are like this now.

Hacking is a good thing.

Not really.

If the fuses are for security, sure. But these aren't.

These fuses are for economic extraction. Doing it the way NVIDIA did it is a safety feature: it ensures that blowing a fuse will never destroy economic value -- it will never turn a card that was previously able to do something correctly into a card that can no longer do that thing correctly.

Product tiers come naturally with IC's, by disabling physical sections of the chip that can't work due to inevitable manufacturing defects. You don't throw defective chips away, you physically cut off those defective sections of the chip, with a saw or with a fuse, so you can sell it without that defect. This works great in the beginning, when your yields are low and you're still dialing in the manufacturing process. But, as yields increase, you might not have enough of the lower tier chips to meet demand. So you, just fuse otherwise good chips. Chips are cheap, after all!

There was probably some cost/engineering justification for it, but there's a very high chance that justification is looong gone. Chips are cheap, but revisions are expensive!

Seems like that would considerably speed up the process, by letting the host continue to run while only the target cycles.

Secondly you never spoke about the squeaky bum moment of actually making the change. Crikey that would have been a long moment.

These findings were disclosed to NVIDIA’s product security team ahead of the publication of this article.

WTF. Does the boot taste that good?

Wow.

6 months ago I looked at used rtx3090s. I marvelled they cost a little over what I paid over a year ago. I ordered 2 more for $900 if I remember correctly.

3 months ago the cheapest used one was $1300, a month ago you could get it for $1500 now it is $1650.

I thought the prices would drop once they reach the peak. But the peak seems to be far away.

A 5090 is only twice faster in typical inference, but costs $10k now. Insanity.

Isn't it like a bit of solder? Or is the problem that the solder would have to be a nanometer across?

If you wanted to patch a die the only real means is through a FIB (Focused Ion Beam) workstation, and even then you need the die structure to allow it (you have to mill your way into the chip and can't have any functional layers in between you and the patch you want to makey), and even when doable it would be an insanely expensive, time-consuming and likely fragile project.