Re:Dalai: Automatically install, run, and play with LLaMA on...
by boredemployee on Sunday March 12, 2023 @10:51PM
(#35130349)
thanks for explaining! How much GPU memory would work nice with 68B?
[ Parent
| on HN ]
Re:Dalai: Automatically install, run, and play with LLaMA on...
by ukd1 on Monday March 13, 2023 @01:03AM
(#35131317)
they said 2g per 1 billion....and it's called 68B...I presume that's 68 billion... 68*2...so at least 136g?
Re:Dalai: Automatically install, run, and play with LLaMA on...
by vishal0123 on Monday March 13, 2023 @04:37AM
(#35132789)
68/2, not 682
Re:Dalai: Automatically install, run, and play with LLaMA on...
by boredemployee on Monday March 13, 2023 @05:16AM
(#35133100)
So, if I understand correctly, that's what you need to run the best model?
With GPU:
VRAM + RAM >= 68/2
Without GPU:
RAM >= 68/2