News for hackers. Stuff that matters.
Re:TinyLlama: An Open-Source Small Language Model by rnd0 on Friday January 05, 2024 @04:56PM (#38885498)
>I wouldn't use either of them for RAG

What's RAG?

Re:TinyLlama: An Open-Source Small Language Model by sp332 on Friday January 05, 2024 @06:38PM (#38886600)
Since the models have a limited context size, you pre-process a bunch of data that might be related to the task (documentation, say) and generate a semantic vector for each piece. The when you ask a question, look up just the few pieces that are semantically most simlar and load them into the context along with the question. Then the LLM can generate a new answer with the most relevant pieces of data.
Re:TinyLlama: An Open-Source Small Language Model by andy99 on Friday January 05, 2024 @04:58PM (#38885535)
Retrieval augmented generative, basically giving it some text passage and asking questions about the text.
Re:TinyLlama: An Open-Source Small Language Model by dmezzetti on Friday January 05, 2024 @05:09PM (#38885667)
If you want more on RAG with a concrete example: https://neuml.hashnode.dev/build-rag-pipelines-with-txtai