Re:Releasing Persimmon-8B
by YetAnotherNick on Thursday September 07, 2023 @03:43PM
(#37424543)
Ablations are important because they tell why the model is better. Here the model is of similar size of llama 7b, trained on 1/3rd the dataset still their claim is that the performance is better. Now this could happen due to lot of things like relu squared or better dataset or 16k tokens. We just don't know why it performed better.
[ Parent
| on HN ]
There are no comments on this comment yet.