News for hackers. Stuff that matters.
Re:Fine tune LLAMA3 on million scale dataset in consumer GPU... by sa-code on Thursday April 25, 2024 @12:41AM (#40153495)
Thank you for saying this! The number of people that would need to fine tune vs just using RAG is really small. People that are not familiar with the source often jump to fine tuning as an option
Re:Fine tune LLAMA3 on million scale dataset in consumer GPU... by Foobar8568 on Thursday April 25, 2024 @01:12AM (#40153688)
I am still unsure where to stand on this fine tuning vs rag. I feel that for live data, rag would be preferable but for daily/weekly updated one, then fine tuning.

Another aspect where I am unsure is multi user for a model e.g. can we have concurrency for a model or the queries have to be queued.

Re:Fine tune LLAMA3 on million scale dataset in consumer GPU... by bigfudge on Thursday April 25, 2024 @03:27AM (#40154530)
Fine tuning doesn’t ’add content’ the way RAG does though. They’re not really comparable in that way.
Re:Fine tune LLAMA3 on million scale dataset in consumer GPU... by Foobar8568 on Thursday April 25, 2024 @06:53AM (#40155859)
So more to be optimized for specific tasks in a domain ?