News for hackers. Stuff that matters.

A comment on Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Custom Models. Parent comment

Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by zby on Friday August 11, 2023 @03:24PM (#37092802)
There is an article at the original site about that: https://www.anyscale.com/blog/fine-tuning-is-for-form-not-fa...

Everybody new to this field thinks that he needs finetuning to teach the LLM of new facts. I made the same mistake initially, later I published a slightly ranty post on that: https://zzbbyy.substack.com/p/why-you-need-rag-not-finetunin...

[ Parent | on HN ]
Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by sandGorgon on Friday August 11, 2023 @03:45PM (#37093104)
Quick question - Gorilla paper talks about finetuning for RAG. Do you see this in practice ? can you do finetuning that specifically affects RAG ?
[ Parent | on HN ]
Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by zby on Saturday August 12, 2023 @08:39AM (#37099576)
Sorry - myself I don't have much experience yet, I am at the research phase, but from what I read it makes sense to finetune the model to better understand the format used for calling the external tools including a search engine.
[ Parent | on HN ]