Ask HN: Anyone Succesfully fine-tuning LLMs?

Posted by Mythli on Friday November 14, 2025 @04:30AM
from the ask-hn dept.

Has anyone of you successfully fine-tuned a LLM?

I have done several attempts for simple use cases and the result was always extremely poor generalization.

Any experience, guides, examples would be valuable.

( 2 comments | 3 points | on HN )

Re:Ask HN: Anyone Succesfully fine-tuning LLMs? by mpcsb on Friday November 14, 2025 @04:46AM (#45925396)

I have done my share of fine tuning on open source LLMs (e.g. Llama). I'm surprised you have very poor generalization.

I assume you're using standard techniques, like lora/qlora, which might leave room for issues with your data. Can you share more details on what is the format of your data points? like, Q/A, free text,...

[ Parent | on HN ]

Re:Ask HN: Anyone Succesfully fine-tuning LLMs? by bob1029 on Friday November 14, 2025 @05:01AM (#45925482)

I've tried it a few times without much success. I think it takes more data and discipline than most are prepared for.

RAG is a lot easier to reason about and much cheaper to iterate.

[ Parent | on HN ]