Re:2022 Letter
by amelius on Friday December 30, 2022 @05:34PM
(#34190912)
Interesting. How are these indexes stored and how are they fed into the transformer model so that GPT can use them? Does this require an additional training step?
[ Parent
| on HN ]
Based on my reading of the docs (I haven't looked at the code), it appears that it uses the existing langchain[1] (which I have used and is excellent) library for constructing longer prompts through a technique known as prompt chaining. So for example, summarizing long documents would involve a map/reduce style effort where you get the LLM to summarize chunks of the document (with some degree of overlap between chunks) and then getting the model to summarize the summaries.
For answering "queries", it appears like it iterates over the documents in the store, i.e., NOT using it like an index, and feeding each document as part of the context into the LLM.