Re:Llama.cpp 30B runs with only 6GB of RAM now
by abhimskywalker on Saturday April 01, 2023 @12:16AM
(#35397063)
"The recent change also means you can run multiple LLaMA ./main processes at the same time, and they'll all share the same memory resources."
So this could have a main and multiple sub-worker llm processes possibly collaborating while sharing same memory footprint?
[ Parent
| on HN ]