News for hackers. Stuff that matters.

A comment on Baidu's Improving Retrieval Augmented Language Model with Self-Reasoning. Parent comment

Re:Baidu's Improving Retrieval Augmented Language Model with... by a-s-k-af on Thursday August 01, 2024 @08:05AM (#41128311)
I assume you are referring to fine tuning a model here?
[ Parent | on HN ]
Re:Baidu's Improving Retrieval Augmented Language Model with... by Tostino on Thursday August 01, 2024 @09:35AM (#41128920)
You could also just continue pre-training of an existing foundation model. Would still be cheaper by not starting from zero.
[ Parent | on HN ]
Re:Baidu's Improving Retrieval Augmented Language Model with... by a-s-k-af on Thursday August 01, 2024 @09:44AM (#41128987)
The amount of accuracy while doing fine tuning or distillation is usually better than pre-training an existing model, not to mention the graph against the cost.
[ Parent | on HN ]