Re:Llama.cpp 30B runs with only 6GB of RAM now
by datadeft on Friday March 31, 2023 @05:04PM
(#35393627)
I have predicted that LLaMA will be available on mobile phones before the end of this year. We are very close.
[ Parent
| on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now
by terafo on Friday March 31, 2023 @05:06PM
(#35393655)
You mean in contained app? It can already run on a phone. GPU acceleration would be nice at this point, though.
Re:Llama.cpp 30B runs with only 6GB of RAM now
by datadeft on Sunday April 02, 2023 @02:52AM
(#35407867)
I was thinking about the limitation by the mobile HW. Yeah GPU support would be nice.
Re:Llama.cpp 30B runs with only 6GB of RAM now
by rickrollin on Friday March 31, 2023 @05:06PM
(#35393662)
People have actually ran it on phones.