Re:TinyLlama: An Open-Source Small Language Model
by sp332 on Friday January 05, 2024 @06:38PM
(#38886600)
Since the models have a limited context size, you pre-process a bunch of data that might be related to the task (documentation, say) and generate a semantic vector for each piece. The when you ask a question, look up just the few pieces that are semantically most simlar and load them into the context along with the question. Then the LLM can generate a new answer with the most relevant pieces of data.
Re:TinyLlama: An Open-Source Small Language Model
by andy99 on Friday January 05, 2024 @04:58PM
(#38885535)
Retrieval augmented generative, basically giving it some text passage and asking questions about the text.
Re:TinyLlama: An Open-Source Small Language Model
by dmezzetti on Friday January 05, 2024 @05:09PM
(#38885667)
If you want more on RAG with a concrete example: https://neuml.hashnode.dev/build-rag-pipelines-with-txtai
What's RAG?