Re:Finetuning Large Language Models
by lumost on Saturday April 22, 2023 @10:11AM
(#35666778)
With search, you don’t know if the model/search engine will retrieve the right context. With fine tuning you don’t know if it will forget important info, or learn incorrect things.
[ Parent
| on HN ]
Re:Finetuning Large Language Models
by kordlessagain on Saturday April 22, 2023 @10:49AM
(#35667108)
With search, you can "tune" the query by using related references to the documents.
Ideally, we include documents that are relevant to the topics at hand and eliminate documents that aren't. Using prompt chaining, or an iterative process of labeling and extracting features, we may be able to increase the efficiency of the final prompt.
Obviously this increases costs, but so does repeated fine-tuning.