News for hackers. Stuff that matters.

A comment on Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?. Parent comment

Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by stevenhuang on Monday December 25, 2023 @11:59AM (#38764126)
> What absolutely does not work is trying to just feed a set of documents into fine tuning.

Not quite. It does work, albeit likely not optimal.

See https://github.com/bublint/ue5-llama-lora

[ Parent | on HN ]
Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by ilaksh on Thursday December 28, 2023 @10:29AM (#38794352)
Have you tried that? I find the results hard to believe because he says he is using Llama 7b and asking it questions but that is not a chat model. Also he does not appear at all in the issues when people ask him about reproducing the results. Instead there are people in the issues recommending RAG or creating a QA dataset.
[ Parent | on HN ]