News for hackers. Stuff that matters.
Re:Dalai: Automatically install, run, and play with LLaMA on... by Tepix on Monday March 13, 2023 @12:29PM (#35138026)
You can't, it needs around 40GB of RAM.

Technically you can by swapping to disk but it would be too slow to be usable.

What you can do however is use the 7B model with 4bit quantization and use it within 8GB RAM.

There are no comments on this comment yet.