Re:Falcon 40B LLM (which beats Llama) now Apache 2.0
by MacsHeadroom on Friday June 02, 2023 @07:05AM
(#36163597)
One benefit of finetuning larger models, like 65B, is to free up limited context space vs few-shot prompting.
If you want a specific kind of interaction with the model then you could take up 1/3rd of the 2048 token context window with few-shot or you could simply finetune it with QLoRA for a few hours on a consumer GPU and then get to use the full 2048 context with the finetuned model.
If you want a specific kind of interaction with the model then you could take up 1/3rd of the 2048 token context window with few-shot or you could simply finetune it with QLoRA for a few hours on a consumer GPU and then get to use the full 2048 context with the finetuned model.