Note that modern RAM protocols assume that multiple bits are in the transmission line that is the wires connecting your CPU to RAM.

Electrical Engineers since at least DDR2, probably earlier, need to ensure that all lines are delay matched to about 100picoseconds.

That is, in DDR2, if DataBit#1 takes 1.1nanoseconds from start-of-wire to end-of-wire, then all other bits must be somewhere between 1.0ns to 1.2ns in length.

This requires impedance controlled pcbs from the manufacturer, and length tracking software for PCB design.

-------

The dielectric of FR4 (the glass/resin used to make PCBs) is what determines the speed of light of the copper actually. And it can be 3.6 or 4.2 or whatever, but the PCB manufacturer will tell you in the PCB specsheets.

-------

I think I heard that DDR4 is now 10picosecomds (10x more accuracy in delay matching), which is doable with modern CAD and PCB software. I dunno the delay requirements of DDR5.

Long long time ago, when Ethernet used coax cables, the max. length of the segment was not because of the signal strength. It was because with the max. length of an Ethernet segment the first half of the ethernet data packet was on the cable, if now the computer on the other side started to send a packet too, a collision would not be detected, and the packet would be lost.

You're conflating latency vs. bandwidth.

It is entirely possible to have many bits in transit on a single channel, wired or wireless.

The article seems to reference having external AI etc accelerators colocated with the RAM they use, but then tied to a host machine via CXL. So that makes a certain amount of sense. Latency sensitive stuff happens co-located with the RAM, and then you just take advantage of throughput back to the host.

I'm personally now imagining a specialized database appliance which takes the role of the whole of the pager and buffer pool management from a DB (or KV store or whatever); a physical box which ties secondary storage arrays + large quantities of RAM + buffer pool mgmt firmware together on a box, then connect to host system via CXL. Host system does query planning end execution and everything else...

Is anybody doing this? Does anybody want to found a startup with me to do this? <sips more and more coffee...>

There are enterprise database offerings that do something like that with RDMA. I'm almost certain CXL is being looked at for improvements.

I just finished my master’s thesis on database buffer management with CXL [1]. It’s not published due confidential hardware information, but we are working on a paper.

[1] https://hpi.de/rabl/teaching/master-theses/ongoing-masters-t...

Very cool. I look forward to the paper!

For some reason the whole issue of buffer mgmt is something I nerd out on a bit.

DRAM latency is tens of clock cycles. There's a minimum of about 10ns of latency happening within the DRAM chip itself, so a slightly longer bus doesn't affect total latency by much.

Wikipedia is showing a cycle time for CAS of less than 1ns for modern RAM

https://en.wikipedia.org/wiki/CAS_latency#Memory_timing_exam...

The figures under 1ns are not latency numbers but merely the duration of a clock cycle.

This is not exactly the reason, but trace length/loss is a major problem.

Laptop SODIMMs are already severely limiting DDR5 bandwidth, server/desktop DIMMs are getting close.

Users aren't gonna like it, but unswappable packaged RAM is coming to most CPUs.

What do you mean by packaged RAM and what protocol does it use that would be faster than DDR5?

> What do you mean by packaged RAM

Depends. Apple uses LPDDR5X on a cheap substrate, which is basically just RAM chips soldered to a tiny motherboard, and that is already very fast. But it could also mean HBM on an expensive interposer or cheaper Intel EMIB, or something like Samsung's proposed Wide I/O, or even something new like stacked RAM with TSVs.

CPU/RAM packaging is getting increasingly complicated.

> what protocol does it use that would be faster than DDR5

Depends, but it can just be straight up DDR5. SODIMMS are terrible because they need 1.35V (vs 1.1V stock DDR5) for really slow speeds and terrible ram timings, while soldered DDR5 and LPDDR5X do not. The closer the DDR5 gets to the CPU, the faster, more power efficient and lower latency it gets.