News for hackers. Stuff that matters.
Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by zamadatix on Wednesday August 13, 2025 @07:07PM (#44894945)
"3 generations" seems like a bit of a stretch. Millions of Blackwell systems use PCIe 6.x today, PCIe 7.x was finalized last month, and this is an announcement work on PCIe 8.0 has started for release in 3 years. I.e. it has only been one month of being one generation behind the latest PCIe revision.

It'll be interesting if consumer devices bother trying to stay with the latest at all anymore. It's already extremely difficult to justify the cost of implementing PCIe 5.0 when it makes almost no difference for consumer use cases. The best consumer use case so far is enthusiasts who want really fast NVMe SSDs in x4 lanes, but 5.0 already gives >10 GB/s for a single drive, even with the limited lane count. It makes very little difference for x16 GPUs, even with the 5090. Things always creep up over time, but the rate at which the consumer space creeps is just so vastly different from what the DC space has been seeing that it seems unreasonable to expect the two to be lockstep anymore.

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by _zoltan_ on Wednesday August 13, 2025 @07:25PM (#44895078)
either I'm misinformed in which case is like to see proof, or you are, but GB200 is not Gen6, just Gen5 and this will only be rectified with GB300 which, while exists in the wild, is not in the millions.

so indeed the parent commenter would be correct that everything is Gen5 right now.

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by zamadatix on Wednesday August 13, 2025 @07:42PM (#44895205)
I'm going off of https://developer.nvidia.com/blog/nvidia-gb200-nvl72-deliver... but I have not actually deployed a GB200 system yet myself. Consumer Blackwell is definitely 5.0 though.
Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by _zoltan_ on Wednesday August 13, 2025 @07:51PM (#44895271)
that article is from last March. the plan was indeed for the GB200 board to support gen6 but it didn't happen and GB300 will rectify it.

at least that's my best educated guess, looking at supermicro's public spec sheet and that it's shipping with CX7 which is Gen5 and not with CX8.

B200 supports Gen6 there is just nothing that would let it run at Gen6.

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by zamadatix on Wednesday August 13, 2025 @07:59PM (#44895323)
At least the GB200s OpenAI purchased have CX-8s, but it's very possible (perhaps even very likely) most of the millions of GB200s were with PCIe 5.0 and CX-7s given the SuperMicro spec sheet.

Edit: found the SemiAnalysis post saying as much https://x.com/SemiAnalysis_/status/1947768988467138645/photo...

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by _zoltan_ on Thursday August 14, 2025 @07:35AM (#44899194)
thanks! very interesting that they got a custom SKU.
Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by jsolson on Wednesday August 13, 2025 @10:49PM (#44896265)
Datacenter Blackwell is Gen6, which is critical when pairing it with CX8 (2x400G) as otherwise you'd be stranding NIC BW.
Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by _zoltan_ on Thursday August 14, 2025 @07:33AM (#44899187)
please link me a B200 (not a B300) system with CX8 that uses gen6 x16 instead of gen5 x32.

to the best of my knowledge this does not exist, but I'd be happy to stand corrected.

(the official NVIDIA DGX B200 is gen5).

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by Melatonic on Wednesday August 13, 2025 @08:26PM (#44895473)
CPUs themselves only have so many PCI-E lanes though right? Wouldnt it make sense (even for consumers) to have peripherals using less lanes (but more speed per lane) for a multi GPU system or something that uses a lot of drives?
Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by zamadatix on Wednesday August 13, 2025 @08:51PM (#44895610)
More lanes = more cost

Faster lanes = more cost

More faster lanes = lots more cost

The chipset also strikes some of the balance for consumers though. It has a narrow high speed connection to the CPU but enables many lower speed devices to share that bandwidth. That way you can have your spare NVMe drive, SATA controller, wired and wireless NICs, sound hardware, most of your USB ports, your capture card, and some other random things connected over a single x4 to x8 sized channel. This leaves the high cost lanes for just the devices that actually use them (GPU and primary, possibly secondary, storage drive). I've got one consumer type Motherboard with 14 NVMe drives connected, for example, just not at full native speed directly to the CPU.

You're just SoL if you want to connect a bunch of really high bandwidth devices simultaneously (100 Gbps+ NICs, multiple GPUs at full connection speed, a dozen NVMe drives at native speed, or similar) because then you'll be paying for a workstation/server class platform which did make the "more faster lanes" tradeoff (plus some market segment gouging, of course).

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by vladvasiliu on Thursday August 14, 2025 @02:33AM (#44897357)
One issue is that, at least on cheaper mobos, these don't work as a "total bandwidth budget" situation. And, especially with newer generation PCIe, it can be a bit frustrating.

Many mobos will operate the available slots such that the total number of active lanes is split between them. But if you use older-generation cards, you'll only get a fraction of the available bandwidth because you're only using a fraction of their lanes, although the physical lanes are physically present.

What I'm thinking about is something like, say, a pair of Gen3 NVMe drives that are good enough for mass storage (running in RAID-1 for good measure) and some cheap used 10 Gb NIC, which will probably be 8x Gen2, all running on a Gen4+ capable mobo.

And, while for a general-purpose setup I can live with splitting available BW between the NIC and the GPU (I most likely don't care about my download going super fast while I game), the downloads will generally go to the storage, so they must be fast at the same time.

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by zamadatix on Thursday August 14, 2025 @12:00PM (#44902047)
Those MBs are cheaper precisely because supporting this kind of bandwidth breakout adds cost (-> a fancier PCIe switch in a higher end chipset/southbridge). If you add the support to do this to them you end up with the more expensive motherboard. Some of the highest end motherboards actually have 2 chipsets/PCIe switches, more cost but a more bandwidth sharing.

You can also buy external PCIe switches (just make sure you're not accidentally buying a PCIe bifurcation device). Most of the time it's cheaper to just buy the higher end motherboard though, e.g. I don't want to know what price "Request a quote" for this PCIe switch which can do x8 4.0 upstream and then quad 4x 3.0 downstream https://www.amfeltec.com/pci-express-gen-4-carrier-board-for... is. I do have a few 3.0 era cards which were more reasonably priced though https://www.aliexpress.us/item/3256801702762036.html?gateway... and they've worked well for me.

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by vladvasiliu on Thursday August 14, 2025 @01:40PM (#44903352)
How high-end are we talking about? Do you know off-hand of any models supporting this?

I haven't seen such features on boards under 200 EUR, from Asus, Asrock and Gigabyte.

The thing is, if I have to splurge for some 400 EUR "gaming" model, I might as well move to a "workstation" CPU supporting more lanes out of the box, and the mobo will be priced roughly the same.

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by michaelt on Wednesday August 13, 2025 @08:50PM (#44895604)
> "3 generations" seems like a bit of a stretch. Millions of Blackwell systems use PCIe 6.x today

True. And yet, if you buy an RTX 5090 today, costing $2400 and released in January this year, it's PCIe 5.0 x16

Re:PCIe 8.0 Announced by the PCI-Sig Will Double Throughput ... by zamadatix on Wednesday August 13, 2025 @08:57PM (#44895644)
Does the second half not already talk to this? Adding on to it further, GN found a few % performance difference on a 5090 going from PCIe 3.0 to 5.0 https://gamersnexus.net/gpus/nvidia-rtx-5090-pcie-50-vs-40-v... meanwhile many DC use cases find PCIe 5.0 unusably slow, driving alternative (non-PCIe) interconnects.