News for hackers. Stuff that matters.

A comment on Mistral NeMo. Parent comment

Re:Mistral NeMo by Bumblonono on Thursday July 18, 2024 @12:02PM (#40996996)
It fits a 4090. Nvidia lists the models and therefore i assume 24gig is min
[ Parent | on HN ]
Re:Mistral NeMo by michaelt on Thursday July 18, 2024 @12:35PM (#40997316)
A 4090 will just narrowly fit a 34B parameter model at 4-bit quantisation.

A 12B model will run on a 4090 with plenty room to spare, even with 8-bit quantisation.

[ Parent | on HN ]