News for hackers. Stuff that matters.

A comment on Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?. Parent comment

Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by sroecker on Monday December 25, 2023 @03:05AM (#38760752)
We just held a workshop about this a few weeks ago: https://red.ht/llmappdev We created a simple chatbot using local models with Ollama (llamacpp), LlamaIndex and streamlit. Have a look at the streamlit folder, it's super easy.

I used this simple example to teach about RAG, the importance of the system prompt and prompt injection. The notebook folder has a few more examples, local models can even do natural language SQL querying now.

[ Parent | on HN ]
Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by 3abiton on Monday December 25, 2023 @07:20AM (#38762017)
Llamaindex has so mucu potential. Any benchmarks on performance compared to fine-tuning?
[ Parent | on HN ]
Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by sroecker on Monday December 25, 2023 @11:22AM (#38763807)
You probably don't need fine-tuning, at least if it's just new content (and no new instructions). It may even be detrimental, since LLMs are als good at forgetting: https://twitter.com/abacaj/status/1739015011748499772
[ Parent | on HN ]
Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by agilob on Monday December 25, 2023 @11:00AM (#38763640)
looks very promising, do you plan to keep this single repo up to date as new things are released?
[ Parent | on HN ]
Re:Ask HN: How do I train a custom LLM/ChatGPT on my own doc... by sroecker on Monday December 25, 2023 @11:19AM (#38763782)
Good question, as you can see I haven't touched it for a month. I wanted to show what's possible then with open source and (open) local models and there's already so much new stuff out there.

I'll probably fix some things this week and then either update it or start from scratch. Guided generation, structured extraction, function calling and multi-modal are things I wanted to add and chainlit looks interesting.

[ Parent | on HN ]