Re:Llama.cpp 30B runs with only 6GB of RAM now
by revelio on Saturday April 01, 2023 @01:57PM
(#35402387)
Native library interfacing isn't really Python's strong suit, interpreter plugins are quite painful to write.
.NET has P/Invoke which is much nicer.
JVM is getting Panama+jextract, which is the nicest yet. You can go straight from header files to pure Java bindings which don't need any extra native code at all. But it's not shipped yet :(
Re:Llama.cpp 30B runs with only 6GB of RAM now
by baq on Saturday April 01, 2023 @03:18PM
(#35403179)
Python has had cffi since figuratively forever, so I’m not sure why you compare native modules to P/Invoke?
Re:Llama.cpp 30B runs with only 6GB of RAM now
by revelio on Sunday April 02, 2023 @06:05AM
(#35408954)
Most important libraries with native components are using C extensions, not cffi.
Re:Llama.cpp 30B runs with only 6GB of RAM now
by alanfranz on Saturday April 01, 2023 @08:36PM
(#35405781)
What is an “interpreter plugin?” Writing a Python C extension is not that painful, it’s quite well supported. And you’ve got cffi and ctypes as well.
Re:Llama.cpp 30B runs with only 6GB of RAM now
by nequo on Sunday April 02, 2023 @06:58PM
(#35416211)
> Ocaml is very niche, I feel it’s an hard sell for a general purpose language. Haskell, 3x that.
The impression about Haskell’s nicheness compared with OCaml prevails. But Haskell has a larger userbase and a larger library ecosystem than OCaml.
Re:Llama.cpp 30B runs with only 6GB of RAM now
by alanfranz on Friday April 07, 2023 @03:47AM
(#35479230)
A few years have passed since I last tried out both languages. Ocaml was sort of approachable, while Haskell required quite a different mindset imho, hence the “nicheness” from the general usage standpoint.
Ocaml is very niche, I feel it’s an hard sell for a general purpose language. Haskell, 3x that.
JS and TS, could be. But are they so much better than Python, if better at all?