Baidu's Improving Retrieval Augmented Language Model with Self-Reasoning

Re:Baidu's Improving Retrieval Augmented Language Model with... by kaspermarstal on Thursday August 01, 2024 @07:21AM (#41128025)

Can anyone explain what is gained by training a model? Why not use the foundational LLM for the relevance, evidence, and trajectory processes?

[ Parent | on HN ]

Re:Baidu's Improving Retrieval Augmented Language Model with... by a-s-k-af on Thursday August 01, 2024 @08:05AM (#41128311)

I assume you are referring to fine tuning a model here?

[ Parent | on HN ]

Re:Baidu's Improving Retrieval Augmented Language Model with... by Tostino on Thursday August 01, 2024 @09:35AM (#41128920)

You could also just continue pre-training of an existing foundation model. Would still be cheaper by not starting from zero.

[ Parent | on HN ]

Re:Baidu's Improving Retrieval Augmented Language Model with... by a-s-k-af on Thursday August 01, 2024 @09:44AM (#41128987)

The amount of accuracy while doing fine tuning or distillation is usually better than pre-training an existing model, not to mention the graph against the cost.

[ Parent | on HN ]