Its all about how fast the memory is.

960MiB is prodigiously large for such a microscopic chip, but if it "only" gain 3-5 times latency reduction over external DRAM, it's still very far from a proper L3 implementation, and far behind L2.

Make DRAM work on 1Ghz+, and then you will see miracles. Imagine a fully synchronous on-die DRAM that can sit just behind L1, or even be connected to load registers directly.

The problem is that effective frequencies for memory round-trip haven't got up much since nineties. If you work with 100% cache misses, your mem will still be working at effective frequency of around 100 to 200Mhz

> If you work with 100% cache misses, your mem will still be working at effective frequency of around 100 to 200Mhz

I think you mean 20MHz. Current DRAM is abysmal at random access.

Yes, more so like this if you discard how it looks from the electrical side.

Even after bytes arrive to DRAM controller analog side, a lot of things have to happen before the data gets to the register. This accounts for the further five to tenfold increase in round trip latency

Oh no, it's paging to ram.

Maybe there’s an inflection point where imperative management of the cache is more effective than heuristic management.

I saw Alan Kay speak once, I think at lambda jam 2013. He made a comment that he’s still waiting for mainstream CPUs to allow us to fit an entire interpreter into on-core cache.

Eh? The LuaJIT and K interpreters definitely fit into L1 instruction cache. (And have since well before 2013 probably.) Honestly I would be kind of surprised if most interpreters for reasonably small languages don't fit into icache. Sure Python's not going to any time soon, but most interpreters are really not very large.

You could get Windows NT 4 running in a processor cache these days.

Forth interpreters probably fit into L1 on modern processors.

And we've had 256KB L2 since the Pentium Pro. That's big enough for a Lua interpreter.

What do you mean by imperative? Static control by the program?

I'm guessing u/hinkley meant having an API for the cache, so a compiler could generate instructions to prefetch cache lines (and maybe even keep them hot) rather than depend on automatic (heuristic) cache management.

Just FYI, prefetch instructions are already a thing (at least on x86). At present they're only hints to the underlying heuristic system though. (https://www.felixcloutier.com/x86/prefetchh)

Cache-as-RAM is also a thing, and allows the sort of pinning you described. Pretty specialized use cases though. (https://stackoverflow.com/questions/41775371/what-use-is-the...)

Read access patterns matter more than cache sizes, triple digit improvements are possible of you have linear reads.

Late Xeon Phis had 16GB of HBM around the die. It could either complement or mirror main memory.

Hasn't Intel bought the company that was on the brink of producing those CPU-memory combos? Unfortunately, I haven't been able to find the name of the company or an article about it.

You mean UpMem or Venray Technology?

Sorry but I don't remember the name at all.