Until LLMs deal with context as a graph and not just a linear order of vectors, it won't matter the amount of context you shove down it's processors, it's always going to suffer from near-sighted processing of the last bits. To generate true intelligence it needs to be able to jump to specific locations without the interceding vectors affecting it's route.