Re:RedPajama (Open LLaMA Replication) 3B and 7B models released
by regularfry on Saturday May 06, 2023 @09:34AM
(#35841218)
Yes and no. Some of the optimisation techniques that are being researched at the moment use the output of larger models to fine-tune smaller ones, and that sort of improvement can obviously only be one-way. Same with quantising a model beyond the point where the network is trainable. But anything that helps smaller models run faster without appealing to properties of a bigger model that has to already exist? Absolutely yes.
[ Parent
| on HN ]
There are no comments on this comment yet.