However, cache is now huge. So, it's latency is usually less of an issue than sequential reads/writes and total size.

You're point isn't invalid but it's sort of misapplied. In fact the latency of DRAM on modern systems is dominated almost entirely by precharge time within the memory IC, which is about an order of magnitude slower than signal propagation. And it's not changing, either. DRAM cycles have been sitting at around 60 MHz for a decade and a half. We're dropping voltage as we shrink the cells, and so there's no net increase. Realistically signal propagation will never be the limit.

Anyway, if you actually trace the furthest bit of a ram chip to the CPU it's a longer path than you might think > 1 foot but < 3 feet.

As to my point tCL is a round trip latency and actually not that far from optimal. DDR is not designed for pure random access so much as cheap access to lot's of ram so yes there are many trade offs, but they are more reasonable when your close to hard limits.

Here is a picture, from the center of the CPU, to the bottom edge of the furthest part of the DIM then up. http://harddiskdirect.com/media/catalog/product/cache/1/imag...

PS: A DIMM is just over 5 inches wide and 0.7 inches tall so if was centered physically on top of a CPU that's 3 inches right there.

But, that's part of the CPU packaging, a signal needs to get to these pins on the bottom of the CPU and that takes time.

My suspicion is that amplifying the differential voltages produced by femtocoulombs of charge at 100% reliability is a harder problem than moving the DRAM chips closer to the CPU, and that speed-of-light has gotten the blame in "pop architecture" due to sloppy overgeneralization.

    2*2 feet is long for a memory bus, but it's partially cancelled
    out by the high velocity factor (sorry, "speed of electricity")
    of 0.951 in parent's calculations. Instead, I'm going with 0.5*2
    feet at a velocity factor of 0.5 for a 2ns roundtrip propagation delay.

On the other hand sequential access bandwith is limited almost entirely by the external interface and has more to with economical factors (pin count is major part of IC price) than any physical limitations.

First word latency: DDR3 SDRAM 6.37ns = 1/156,985,871 of a second.

DDR is not really setup for random access, because it's setup for talking to L2/L3 cache not registers. tRAS ~= tCL + tRCD + tRP which kind of but not exactly exactly what you want for random access.

I don't understand your argument that the L2 and L3 caches tip the balance in favor of <#CAS>. If anything, I'd expect them to do the opposite, because the SRAM and DRAM both aim to exploit locality, meaning that better L2 and L3 caches would reduce #CAS faster than #RAS. Of course, armchair reasoning about such complex systems as memory controllers and caches can only go so far, so I wouldn't be shocked to be wrong, but I would certainly demand actual statistics before changing my view.

If you look at CAS latencies [1] then it's clear that memory speeds stagnated a long time ago:

[1] https://en.wikipedia.org/wiki/CAS_latency

This is a central reason why the cache size per thread has not changed much in the last 20 years (beside of course fabbing/yield limits). (And because caches are subject to a similar effect as Dennard scaling)

(btw. I wonder if it'd have been smarter by the memory industry if they specified latencies in absolute numbers (or relative to the memory array clock, not the bus clock), not bus cycles, since the rising latency numbers have been a marketing issue in every generation.)

Also, AMD for example uses stacked ram (HBM), but that needs very special packaging and active cooling. http://www.amd.com/en-us/innovations/software-technologies/h... https://en.wikipedia.org/wiki/High_Bandwidth_Memory

I doubt we will see a drastic improvement in DRAM latency unless the underlying process changes a lot (i.e. different physical storage mechanism or very different manufacturing).