News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now. Parent comment

Re:Llama.cpp 30B runs with only 6GB of RAM now by nunobrito on Saturday April 01, 2023 @08:41AM (#35399833)
Still a poor excuse. Had they written this in Java and things wouldn't be so difficult both on performance and maintainability.

Never understood why people think that indented languages are any simpler when in fact they bring all kinds of trouble for getting things done.

[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by revelio on Saturday April 01, 2023 @10:50AM (#35400733)
There's a Java ML library called Tribuo that might be worth looking at.
[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by dopidopHN on Saturday April 01, 2023 @11:06AM (#35400868)
Thanks, the boring aspect of Java is appealing here.
[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by rfoo on Saturday April 01, 2023 @03:26PM (#35403267)
There's deeplearning4j (from Theano days!), go figure why it didn't take off.
[ Parent | on HN ]