News for hackers. Stuff that matters.
Re:GPT-3.5 Turbo fine-tuning and API updates by davedx on Wednesday August 23, 2023 @04:22AM (#37233237)
That doesn't make sense - if you're doing retrieval augmentation, then every prompt will be different depending on the query, right? So you still need to burn those tokens.
Re:GPT-3.5 Turbo fine-tuning and API updates by danielbln on Wednesday August 23, 2023 @04:37AM (#37233317)
You can persist the knowledge you would normally retrieve dynamically and inject into the context at runtime via finetuning directly into the model weights. This means you won't have to pull in a glob of additional data with every prompt, but only need to push the query through, which can be answered by the data persisted in the weights.

However, as others have pointed out, you lose the ability to reference sources reliably and the output is much more prone to hallucination.

Re:GPT-3.5 Turbo fine-tuning and API updates by davedx on Wednesday August 23, 2023 @04:52AM (#37233403)
Ahh understood, thank you.