So, ~1/2 of your total latency (~5 or 6 of 10 cycles) is just waiting electricity to travel down wires.
Anyway, my point is when physics puts hard limits in place cost / benefit trade-offs tend to be centered around that limit. http://en.wikipedia.org/wiki/Amdahl%27s_law
Note: This is purely a discussion about latency which is what matters for Linked List access. You can send a lot of data as part of a single request cycle, but that adds to your effective latency.
That doesn't seem right.
http://www.bhphotovideo.com/bnh/controller/home?O=&sku=10887...
Looks like it might be 1ns away at .5c, which is 10% of the minimum latency and 3% of maximum (excluding refresh) latency -- and those are latencies I took off a top-of-the-line gaming RAM (my own computer is 11-11-11). It also looks like the RAM could trivially be moved closer if there were an advantage to doing so (e.g. square, stacked layout).
My point is not that the limits aren't physical, it's that the limiting physical factors are currently in silicon designs, not FR4 layouts.
At .25c 1ns = Less than 3 inches. A single DDR3 DIMM is ~6 inches wide and ~1 inch tall so that's not long enough to cover a single DIMM.
Look at that picture again, including clips and those dimm slots are 7 inches wide. The top edge of a DIMM is 1 inch over the MB. And the furthest dimm is ~2 inches further from the CPU. The trace from the CPU's on board memory controller to the MB is ~1 inch. And the CPU is ~ 2 inches from the DIMMS.
EDIT: http://en.wikipedia.org/wiki/DDR3_SDRAM#mediaviewer/File:4GB... The DIMMs are not strait paths so that can easily be another inch.
Looks close to 1 foot to me which at .25c that's ~4ns. Or 40% of the minimum latency.
As to layout, that stuff is much harder than you might think as the wires can't go over each other.
For a DDR RAM "speed" of, say, 1866 MHz you have an actual clock of 933MHz and typical timings of roughly tCAS=tRCD=tRAS=9 to tCAS=tRCD=tRAS=11 (measured in cycles of the actual clock). If you're lucky and the correct memory row is already open, then you'll wait 10ns for a memory request to complete, but you will regularly wind up waiting 20 or 30ns if you need to open or precharge+open a row -- and that's assuming that there are no refresh cycles blocking the request. This adds up to a lot of inches when you multiply by 6in/ns, not to mention a lot of CPU die area to optimally order memory requests & refreshes (you can have one in-flight per bank).
You're right that light should be the limiting factor, though. It has long seemed to me that we need to undergo a DRAM->SRAM shift analogous to the HDD->SDD shift. Drop the capacities a bit (factor of ~6 from 6 transistors/cell vs 1 transistor/cell?) in exchange for speed. I'm sure there's a reason why it's not happening (energy efficiency?), but I'm an industry outsider and have no idea what it is. If anyone reading this does, I'd love to hear.