Llama.cpp 30B runs with only 6GB of RAM now

Re:Llama.cpp 30B runs with only 6GB of RAM now by victor96 on Friday March 31, 2023 @05:40PM (#35394041)

Less memory than most Electron apps!

[ Parent | on HN ]

Re:Llama.cpp 30B runs with only 6GB of RAM now by terafo on Friday March 31, 2023 @05:43PM (#35394062)

With all my dislike to Electron, I struggle to remember even one Electron app that managed to use 6 gigs.

[ Parent | on HN ]

Re:Llama.cpp 30B runs with only 6GB of RAM now by baobabKoodaa on Friday March 31, 2023 @06:27PM (#35394582)

I assume it was a joke

[ Parent | on HN ]

Re:Llama.cpp 30B runs with only 6GB of RAM now by mrtksn on Friday March 31, 2023 @05:54PM (#35394174)

I've seen WhatsApp doing it. It start with 1.5G anyway, so after some images and stuff it inflates quite a lot.

[ Parent | on HN ]

Re:Llama.cpp 30B runs with only 6GB of RAM now by xiphias2 on Friday March 31, 2023 @07:23PM (#35395135)

Maybe they should do more mmap as well.

[ Parent | on HN ]

Re:Llama.cpp 30B runs with only 6GB of RAM now by joshbochu on Friday March 31, 2023 @09:25PM (#35396044)

LMAO

[ Parent | on HN ]