News for hackers. Stuff that matters.
Re:PMC-LLaMA: Further Finetuning LLaMA on Medical Papers by Taek on Tuesday May 09, 2023 @02:52AM (#35870984)
75B tokens is not really enough data to make an intelligent model. Llama was trained on over 1 trillion tokens.

And yes, training on top of LLaMA could introduce a lot of unexpected behavior, but that's just where the State-of-the-Art is today

There are no comments on this comment yet.