Re:PMC-LLaMA: Further Finetuning LLaMA on Medical Papers
by kristjansson on Tuesday May 09, 2023 @01:14AM
(#35870423)
AIUI they (1) finetuned LLaMA on 75B tokens of medical papers, and called the resulting model PMC-LLaMA, (2) finetuned that model on task-specifc data, then (3) separately finetuned the base model they started with in (1) on the same task specific data for comparison. So, they're reporting 6 separate conditions: {LLaMA, PMC-LLaMA} x {Zero-shot, Finetune, PEFT}.
Could definitely use some more exposition around that though.
Could definitely use some more exposition around that though.