Re:TinyLlama: An Open-Source Small Language Model
by sp332 on Friday January 05, 2024 @06:38PM
(#38886600)
Since the models have a limited context size, you pre-process a bunch of data that might be related to the task (documentation, say) and generate a semantic vector for each piece. The when you ask a question, look up just the few pieces that are semantically most simlar and load them into the context along with the question. Then the LLM can generate a new answer with the most relevant pieces of data.
[ Parent
| on HN ]
There are no comments on this comment yet.