Re:PCI-SIG Releases PCIe 4.0 Specs
by userbinator on Sunday October 29, 2017 @10:45PM
(#15582796)
PCI-SIG members may download the PCIe 4.0 specification for free at the Specification Library. Non-members may purchase a hard copy of the specification for $4,500.
Insane. PCIe 4.0 v0.3 and 0.9 have leaked already; I wonder how long until v1.0 is available at the usual sites. Looking at the history of the PC, it's sad that key specifications in its design have become so closed.
Re:PCI-SIG Releases PCIe 4.0 Specs
by spamizbad on Sunday October 29, 2017 @11:23PM
(#15582922)
Maybe a traditional "PCI-E card" formfactor, but I don't think there's anything in the specification preventing you from building a 32 or 64-lane device.
Re:PCI-SIG Releases PCIe 4.0 Specs
by kabdib on Monday October 30, 2017 @12:51AM
(#15583160)
PCIe 3.x supports a power-of-two number of lanes up to 16; it handles all the gory stuff needed to aggregate those lanes. Going to more than 16 lanes means having multiple channels, and you'll have to worry about clock skew, delayed (retried) and failed transactions and other wonderful things you get with multiple independent pipes.
I've never done it, but I'm interested in how hard it is. I know a guy who's done detailed design of PCIe hardware; I'll have to ask him.
Re:PCI-SIG Releases PCIe 4.0 Specs
by mechagodzilla on Monday October 30, 2017 @08:41AM
(#15584543)
I’ve done a lot of PCIe hardware development. I think there was some notion of a 32x connection early on, spec wise, but x16 is as wide as it’s ever gone in practice. All high performance PCIe traffic is controlled by DMA hardware on the endpoint, so having an endpoint with multiple logical x16 connections back to the host cpu would be quite straightforward to implement, and you wouldn’t need to worry about any of the gory details you listed. Some FPGAs are available with multiple endpoints embedded in them, for instance.
Re:PCI-SIG Releases PCIe 4.0 Specs
by luckydude on Monday October 30, 2017 @10:19AM
(#15585149)
Could you explain it like I'm five (or really like I'm a software geek and don't know this stuff) how the signals in the multiple lanes are synchronized? Is there some centralized clock that everyone is using?
Re:PCI-SIG Releases PCIe 4.0 Specs
by crzwdjk on Monday October 30, 2017 @01:03PM
(#15586733)
From what I understand, at the physical level PCI-E uses an 8b10b encoding, which allows the clock to be recovered from the data. Each lane's clock might still be skewed with respect to other lanes, but there's some deskew logic that detects and corrects for this skew. I'm not 100% sure when the skew detection happens. It might be negotiated as part of link negotiation, or possibly it gets detected at the start of a packet, which has a special 8b10b symbol that's not used for data so it's easy to spot.
Re:PCI-SIG Releases PCIe 4.0 Specs
by kabdib on Tuesday October 31, 2017 @11:22AM
(#15593875)
PCIe 3.0 actually uses a 128b to 130b encoding (I guess clock recovery got a lot better, but they also use different whitening patterns). Data is distributed across the lanes, bit by bit, and reassembled by the receiver. There's a bunch of training and link configuration stuff going on.
Once you have multiple channels of these (because 16 lanes wasn't enough bandwidth) you have the possibility that one channel gets transaction errors while the other doesn't, resulting in skewed arrival or even just missing data. So you need a software layer to deal with this, it's no longer "plug something in and you get bus level reliability," instead it's more like networking, where you have to do some reassembly as data arrives, and probably embed sequence number or other information in the data so you can identify bad and missing transfers. (Or you could say, "screw it, it's reliable enough" and wait for customers to complain; if they never do, you either made the right engineering decision or your business isn't big enough yet :-) ).
Re:PCI-SIG Releases PCIe 4.0 Specs
by p_l on Monday October 30, 2017 @10:35AM
(#15585243)
The signals are skewed in predictable way, which allows recovering the ordering of transmission. IIRC, there's no common clock at all, each lane is "independent" and the ordering is applied on top.
Insane. PCIe 4.0 v0.3 and 0.9 have leaked already; I wonder how long until v1.0 is available at the usual sites. Looking at the history of the PC, it's sad that key specifications in its design have become so closed.
I've never done it, but I'm interested in how hard it is. I know a guy who's done detailed design of PCIe hardware; I'll have to ask him.
Once you have multiple channels of these (because 16 lanes wasn't enough bandwidth) you have the possibility that one channel gets transaction errors while the other doesn't, resulting in skewed arrival or even just missing data. So you need a software layer to deal with this, it's no longer "plug something in and you get bus level reliability," instead it's more like networking, where you have to do some reassembly as data arrives, and probably embed sequence number or other information in the data so you can identify bad and missing transfers. (Or you could say, "screw it, it's reliable enough" and wait for customers to complain; if they never do, you either made the right engineering decision or your business isn't big enough yet :-) ).
PCIe 3 is 128b/130b, and it would be wildly inaccurate to refer to an x16 as a "2048-bit bus."