The problem is that knowledge in a LLM is separated in the mathematical sense (vector direction) but not necessarily meaningfully grouped in the matrix (would be easier to split between disk/memory) if such.

I think a rough analog is that it would be difficult to organize the rows of a hash table of everybody in a country by their geographic location.

At this point, people are just thrilled LLMs can even function as they do…