Re:A comprehensive guide to running Llama 2 locally
by ErneX on Wednesday July 26, 2023 @07:55AM
(#36876207)
No idea, I just said I wanted to try this out and see how it performs.
Doesn’t VRAM amount limit the size of the model you can load? I’m not talking about training just inference. I also pointed out these are not the greatest GPUs available, just that the advantage they have is being able to address more memory since on those machines is a shared block between system and GPU.
Doesn’t VRAM amount limit the size of the model you can load? I’m not talking about training just inference. I also pointed out these are not the greatest GPUs available, just that the advantage they have is being able to address more memory since on those machines is a shared block between system and GPU.