Llama.cpp 30B runs with only 6GB of RAM now

Re:Llama.cpp 30B runs with only 6GB of RAM now by int_19h on Monday April 03, 2023 @05:30PM (#35431561)

I think what OP is saying is that decades-old systems wouldn't have C++11-compatible compilers on them.

[ Parent | on HN ]

Re:Llama.cpp 30B runs with only 6GB of RAM now by bboygravity on Tuesday April 04, 2023 @11:49AM (#35441542)

And maybe that "C++" is now basically a bunch of different incompatible languages instead of just 1 language, depending on what "xx" is (11, 14, 17, 20, 23, etc).

It's like Python 2 vs Python 3 except even worse.

[ Parent | on HN ]

Re:Llama.cpp 30B runs with only 6GB of RAM now by int_19h on Tuesday April 04, 2023 @02:32PM (#35444029)

In my experience, C++03 code works just fine without changes on a C++11 and C++14 compilers, so no, it's not at all like Python 2/3. The few features that were ripped out were exactly the stuff that pretty much no-one was using for good reasons (e.g. throw-specifications).

[ Parent | on HN ]