Re:Mistral NeMo
by imtringued on Thursday July 18, 2024 @04:16PM
(#40999303)
They did quantization aware training for fp8 so you won't get any benefits from using more than 12GB of RAM for the parameters. What you might be using more RAM is the much bigger context window.
[ Parent
| on HN ]
There are no comments on this comment yet.