You just described RAG: augmenting an LLM with external memory. Perhaps the part you are skipping is that the LLM synthesizes the retrieved information with its own knowledge into one coherent whole.

It's abstractive- (new) versus extractive (old) summarization.

What makes it useful is that it does the work of synthesizing the information. Imagine you ask a question that involves bits and pieces of numerous articles. In the past you had to read them all and mentally synthesize them.