> Would it make sense to move the power delivery stuff to a separate, longer-lived part?

That's what your computer's power supply is: A long-lived part that delivers bulk 12V power to be regulated down to local voltages on the spot.

High performance parts will always need local voltage regulation. The VRMs must be physically close to the chip to provide a low inductance path to serve fast current transients. Moving the VRMs too far away from the chip would compromise the power delivery. Running the power through wires or connector pins is significantly worse than running through broad power planes embedded in the PCB.

At these speeds, designers spend a lot of time simulating and analyzing the power delivery network on these PCBs. The power regulation and delivery is fine tuned to support the high speeds required. You can search for "Power Delivery Network Analysis" if you want to learn more.

That reminds me of some hardware teardowns (eg of the new consoles, latest graphics cards and such) where they show how the VRM's are scattered around to balance what they serve as equally as possible (to keep heat spread out?) while still being as close as possible to what they provide power to. Its quite interesting!

This is an awesome comment, thank you! I had long wondered myself why GPU power delivery was so complicated.

Figures you have to look at to be scared about in this space is an RTX 3070 can pull 250 watts and the core is 1.2v (I haven’t confirmed this) then the card can be sucking up an average of 200 amps. Which is a lot. Peak currents can be a lot more than that too.

That's a crazy amount of current to pull. I have melted wires or even welded things together with a super-capacitor setup delivering a similar amount.

My 3090 tops out at 400W on a not even very aggressive overclock. And it makes my room HOT when it's pegged. These graphics chips are insanely power hungry and beefy as hell these days.

As an example of a PC power supply doing things slightly differently, the Lenovo P620 has a power supply that connects to the motherboard with a slot connector. I heard that it only supplies one voltage and the rest of the voltages are generated on the board, but I can’t find a reference for that.

https://www.anandtech.com/show/16482/lenovo-thinkstation-p62...

Intel is introducing a new standard where the PSU just supplies 12V: https://m.youtube.com/watch?v=heyGtgdfN7A&vl=en , but it still has cables. The Lenovo design is neat but it would mean the placement of the PSU relative to the motherboard is unchangeable..

A100 uses such "single part" power supply sitting right next to the package.

I believe such chips can be easily unsoldered relatively easily, and reused.

And by the way, refurbished, and recycles parts, and even passives ARE a thing, it's just the part of the industry you never see in manufacturing of goods going to first world countries, but in China you can buy 2nd hand electronics components quite easily.

The reason why the VRM is tightly integrated with the asic is that there are extremely stringent constraints on the power integrity connecting these two parts. To first order, a decent GPU draws about 1000 amps of current, so even a single milliohm of resistance would absorb the entire voltage supply. Not just the ripple tolerance, the entire power rail! This is before we even get into the ludicrous requirements on transient behavior, inductance, etc, of the connection.

They do make GPUs with a socket for the asic (typically only used in test labs) so it is theoretically possible that you could for example... buy a 2070 carrier and then get nvidia to sell you a 2080 asic to socket into the carrier. But generally the technologies used in GPUs change every generation (memory technology, number of memory channels, display technologies, etc) so you probabably wouldn't be able to keep your carrier more than 1 maybe 2 generations tops.

I think there is a typo in "1000 amps of current."

No, a modern GPU literally draws about 1000 amps of current. For example, a non-overclocked 3090 draws about 500w under load (some of this goes to memory, but that is powered by the motherboard not by the external power connector), at a Vcore of roughly 1.1v which gives a bit over 450 amps--on average! The peak power consumption (as measured on a nanosecond scale being consumed by the asic itself) can easily be double the average power consumption due to the bursty nature of asic power draw.

Remember that it's not just one core - that's what makes a GPU so powerful. My understanding is it's hundreds or thousands of 1.1v cores, which adds up to the 500w.

EDIT - wait, what GPU draws 500w? I can't find one.

Not sure any base cards draw this much. But might be doable via modding:

https://www.notebookcheck.net/The-NVIDIA-GeForce-RTX-3090-ma...

for desktop GPUs 1000 is bit much, but 600-700A are totally doable under LN2

I think people are conflating watts and amps, watts measure power, amps measure flow. Watts = amps * volts. High power usage with low voltages results in high amperage.

Even so, in this very case the amps are higher than the watts(!) as the voltage is below 1V.

If Vcore is near 1 volt, then the magnitude difference between amps and watts is pretty small for the sort of discussion we are h aving

>Would it make sense to move the power delivery stuff to a separate, longer-lived part?

Likely no, the voltage regulation is already hard being that close to the chips and memory. A modern vrm for rtx 3090 is 16 phase by 70A stage. The gpus already have multiple power pins and with so high amps the losses in the conductor become non-trivial, hence voltage regulation is even more difficult.

Moving it farther away would make the regulation extremely conservative, overshooting and delivering too high voltage.

Even if all that happens GPUs would still require massive amounts of filtering caps.

I believe transformer based "VRMs" (they are actually unregulated) like these https://www.techspot.com/images2/news/bigimage/2020/05/2020-... is the future as a way more practical way to deal with power supply of few hundred Amp chips.

16 phase buck controllers are greatly, greatly impractical, and the only reason the industry is stuck with them is that nobody was brave enough to do serious research into making responsive magnetic based DC-DC, while everybody knew that such thing should be possible.

It's interesting curiosity how Taiwanese made it an industry trend out of multi-phase bucks because people who were making first PCs in Taiwan had rather subpar electronics engineering education, or were basically self-taught, and designed computer boards by trial end error back in nineties.

Intel tried to do the exact opposite, move the VRM to the processor die itself, in Haswell. It did not go well because of the extra heat, but those chips make for great overclock testbeds haha.

Other than that, why would you move it to a separate part? Where? Either you push that extra cost onto the motherboard manufacturers or you need a whole new standard and slot for the VRM? What's the advantage?

These days, everything is moving towards tighter integration (fuck soldered CPUs, RAM and SSDs), so no one is going to do that.

We already have 6 and 8 pin additional connectors on graphics cards, so the connector/slot aspect is easily solved.

I think the underlying premise is if you upgrade graphics cards more often or otherwise buy 2-4 graphics cards for every motherboard you buy that moving the VRM to the motherboard (and add costs there), you can still save money and reduce waste.

>We already have 6 and 8 pin additional connectors on graphics cards, so the connector/slot aspect is easily solved.

Those 8 pins are 12V, not 1V. They are spec'd to carry like 12.5A each - make it 15A - bit warmer wires. So around 15W per connector at 1V... and the need is over 300W. So there you have it 20 8-pin connectors.

There is a reason power lanes of the grid are higher voltage - they need a lot less current, so they are efficient. Lower voltage should remain on the PCB with multiple power planes.

To argue alongside your excellent points: it's not just bit warmer wires, but also the underlying voltage drop that would be harmful in such a design, especially in the context of a 1V DC supply.

The advantage is being able to replace the GPU, RAM, etc without having to throw away the VRMs.

Those VRMs are not that expensive, and you can't realistically move them far away from the load without constantly under- or over-voltaging the transistors you're trying to power.

Your power company does not deliver 5 volts to your house for your phone charger, nor does it transport 240 VAC more than a few parcels.

> Making a cheap 400A connector might be somewhat difficult, but not impossible.

It will be, I dare you to do.

Inductance will be a problem. GPU makers are already having troubles with inductance VRM directly near the chip.

> Now, the voltage regulation is pretty darn standard.

Saying this, take a look on A100

https://www.techspot.com/images2/news/bigimage/2020/05/2020-...

No VRM in a conventional sense. It's a fancy HF DC-DC with transformator!

The power delivery part is cheap; it's the chips and memory that's expensive.

There's a reason why monitors aren't sold as two parts: the display itself and the power circuitry.

Some monitors actually are powered from external adapter. No idea how common it is, but one is sitting on my desk.

Interestingly, it's also the one that turns off when there's some interference from the lights turning off in the next room,...

I guess I'll try re-arranging the wires if that stops the pickup of interference.

> Some monitors actually are powered from external adapter. No idea how common it is, but one is sitting on my desk.

I vaguely remember it was for regulatory reasons. I searched around and it looks like it's because with an external power supply they only need to certify the brick rather than the whole unit https://www.reddit.com/r/sysadmin/comments/24jxfj/anyone_els...

It being thinner is also a bonus.

>Some monitors actually are powered from external adapter. No idea how common it is, but one is sitting on my desk.

Well that's just AC/DC which is similar to the PSU of a computer or laptop. The monitor, itself, has voltage regulation for much lower volts that the one connected to the mains.

> Interestingly, it's also the one that turns off when there's some interference from the lights turning off in the next room,...

My Samsung monitor (actually a TV) does this, with any spike/dip on the mains line (eg the fridge turning on). I assumed this was just poor quality power circuitry.

Is yours a Samsung too by chance?

Both my monitors have an external brick. Unless that is something else?

This is exactly the same as the laptops have a power brick, or desktops have a PSU. It's relatively dumb AC/DC switch mode power supply (say ~90-230 to 18VDC) w/o any major care about ripple (100mV ripple is amazing already) or even exact voltage. The monitors do have more voltage regulation inside.

Yes, that reason is easier UL and CE certification.

We already had that, and it was terrible. Socket 5 and some 7 boards shipped with connector for 2.8V VRM. Then then you had server boards with elaborate external VRMs, like in the picture on the wiki https://en.wikipedia.org/wiki/Voltage_regulator_module

I remember dual Pentium Pro motherboards with socketed power regulator modules, one for each processor.

I think a better approach for enterprise and high end hardware would be socketed capacitor or diode banks. Those are the parts that usually deteriorate or fail anyway.

This way you can keep the components close to where they need to be and still get that modularity you're looking for.

What you will gain in reusability you will more than lose in costs for 00 gauge copper wire, space constraints and interconnect costs, and likely the transients would still kill you.

I remember seeing some older video cards, and they actually had removable video memory in some slots on the card itself

Yeah, but the memory in something like a Millenium 2 runs at like 1/500th the speed compared to the memory in a modern GPU (like the GDDR6X in the RTX 30 series cranking out 21 GBit/s per pin). Meanwhile the physics stayed the same.

I’ve heard that about SSDs and EEPROMs. They don’t require a 27V or whatever supply anymore because they’ll generate it on-die with charge pumps.

But charge pumps are inefficient (like 25% efficient), take up a lot of die Space.

It would be better to have a switch mode PS generate the voltages required for erasing (effectively required for every write). On a USB key, whatever, but on a 24/7 system or laptop, it can matter.

Maybe it’s a thing now, but it’s very recent:

https://e2e.ti.com/blogs_/b/powerhouse/posts/how-to-speed-ss...

> It would be better to have a switch mode PS generate the voltages required for erasing (effectively required for every write).

Interestingly, chips like simple USB and Ethernet controllers sometimes do feature an on-die switched-mode DC-DC circuitry to simplify the system design. You just connect an inductor and a capacitor to the DC-DC pin, and route its output to the Vcore input, pretty convenient.

Do you mean this [1] kind of charge pump? The article indicates 90-95% efficient, that's a huge gap from 25%

[1] https://en.wikipedia.org/wiki/Charge_pump

Yes, but the keyword here is "sometimes", charge pumps can be efficient only when very specific conditions are met, usually low current and power (but there are exceptions), efficiency drops sharply as current increases. Putting the circuit onto the silicon creates many additional limitations.

I highly recommend you to read Linear Technology AN104 - Load Transient Response Testing for Voltage Regulators [0] to understand the staggering challenge in hardware engineering to design an acceptable voltage regulator for digital processors. Even though it describes 2006's tech for a simple embedded system - it's basically a toy today, but makes a good example.

The problem is transient response. The power consumption of digital processors varies wildly. For example, if a processor suddenly switches from an idle to active state, a sudden increase of current consumption makes the voltage to fall, until the regulator is able to react by conducting more power, hopefully fast and smooth enough before the processor crashes, on the scale of microseconds or even nanoseconds. This is not a trivial problem. Inductance is another problem - in any closed circuit, every piece of conductor has the tendency to resist the change of current, which means even if the regulator itself is ideal, if it's too far from the processor, it's physically impossible to react fast enough before the output voltage goes out of control, thus all ASIC requires local regulation, and the end result is extremely sensitive to board layouts and components choices.

This note describes how to build a "simulator" to test such transient currents and analyzes various circuits for this purpose - not very readable and you can just ignore this part. Just read page 1, then start reading page 6 ("Capacitor’s Role in Regulator Response") for some actual tests. Page 8 has a practical example: testing a power supply for Intel P30 Embedded Memory.

> supply tolerances are tight. Figure 26’s error budget shows only 0.1 V allowable excursion from 1.8 V, including all DC and dynamic errors. The LTC1844-1.8 regulator has a 1.75% initial tolerance (31.5 mV), leaving only a 68.5 mV dynamic error allowance.

2020 PC hardware is at least two orders of magnitude more complex than that. A simple example can be Altera Corporation AN 574: Printed Circuit Board Power Delivery Network Design Methodology for an FPGA - every single component connected to the power supply (and the circuit board itself) must be modeled and simulated as a resonant circuit with complicated frequency response and impedance characteristics [1].

[0] https://www.analog.com/media/en/technical-documentation/appl...

[1] https://www.intel.com/content/dam/www/programmable/us/en/pdf...

The last page of the Linear Tech application note is certainly not something I expected to see. Glad to see they're having some fun

I wonder what cost savings would be passed along doing it this way. Also, would PSU be a better (or possible) option? A PSU has the ability to outlast a motherboard in the typical consumer cycle, as a motherboard will only see 1 or 2 GPUs before a system upgrade due to chipset/CPU socket/RAM upgrades

The proposed idea is not viable for so high currents with so little ripple and microseconds transient responses.

It happened the same with TV tuner for computers. Standard Philips tuner and radio chips, and custom additions plus a PLL from the factory.