Re:GPT-3.5 Turbo fine-tuning and API updates
by textcortex on Wednesday August 23, 2023 @03:54AM
(#37233030)
Most probably they are doing something called prompt tuning. This creates a small ai model that adds virtual tokens to prompt before passing to original model: https://developer.nvidia.com/blog/an-introduction-to-large-l...
I initially also thought this would be one of the best use cases for fine-tuning (teaching the model new data), but I've seen quite a few people say fine-tuning should not be used to teach the model new data, but more like new formatting and style of response. This blog post seems to concur.
I do wonder how OpenAI does fine-tuning. I'm guessing it doesn't use Lora.