You don’t even solve the problem by moving the memory on-die, as the majority of time is not spent sending the signal on traces, it’s waiting for the relatively slow memory to find, read, and send the data on the bus.

Expanding the cache so everything fits in it is one way to achieve the performance uplift of cache hits, but cache is expensive compared to memory, and current cache sizes are tiny compared to RAM. If cache gets 50x bigger tomorrow, chips will get at least 10x as hot, power-hungry, and expensive.