Accessing RAM is very slow and CPU caches don't scale, though. So it's not as simple as you're presenting it

Well, for general computing I don’t think we have an answer either way. Yeah, for some hot loop with an array we get insane performance, but that is a very specific workload and not applicable everywhere. What about a chat application, which parts of the memory should be physically close to each other?

Also, Java’s GCs are compacting, putting similarly old objects relatively close to each other. The same random chat program in C might use a linked list with worse characteristics so it is really not that obvious to me what would be a good solution.