Re:Llama.cpp 30B runs with only 6GB of RAM now
by alanfranz on Saturday April 01, 2023 @11:31AM
(#35401049)
Any JVM language or .NET language will take more to interface with native libraries, it’s not the same.
Ocaml is very niche, I feel it’s an hard sell for a general purpose language. Haskell, 3x that.
JS and TS, could be. But are they so much better than Python, if better at all?
Re:Llama.cpp 30B runs with only 6GB of RAM now
by revelio on Saturday April 01, 2023 @01:57PM
(#35402387)
Native library interfacing isn't really Python's strong suit, interpreter plugins are quite painful to write.
.NET has P/Invoke which is much nicer.
JVM is getting Panama+jextract, which is the nicest yet. You can go straight from header files to pure Java bindings which don't need any extra native code at all. But it's not shipped yet :(
Re:Llama.cpp 30B runs with only 6GB of RAM now
by baq on Saturday April 01, 2023 @03:18PM
(#35403179)
Python has had cffi since figuratively forever, so I’m not sure why you compare native modules to P/Invoke?
Re:Llama.cpp 30B runs with only 6GB of RAM now
by revelio on Sunday April 02, 2023 @06:05AM
(#35408954)
Most important libraries with native components are using C extensions, not cffi.
Re:Llama.cpp 30B runs with only 6GB of RAM now
by alanfranz on Saturday April 01, 2023 @08:36PM
(#35405781)
What is an “interpreter plugin?” Writing a Python C extension is not that painful, it’s quite well supported. And you’ve got cffi and ctypes as well.
Re:Llama.cpp 30B runs with only 6GB of RAM now
by nequo on Sunday April 02, 2023 @06:58PM
(#35416211)
> Ocaml is very niche, I feel it’s an hard sell for a general purpose language. Haskell, 3x that.
The impression about Haskell’s nicheness compared with OCaml prevails. But Haskell has a larger userbase and a larger library ecosystem than OCaml.
Re:Llama.cpp 30B runs with only 6GB of RAM now
by alanfranz on Friday April 07, 2023 @03:47AM
(#35479230)
A few years have passed since I last tried out both languages. Ocaml was sort of approachable, while Haskell required quite a different mindset imho, hence the “nicheness” from the general usage standpoint.
Re:Llama.cpp 30B runs with only 6GB of RAM now
by xxpor on Saturday April 01, 2023 @01:10PM
(#35401969)
Outside of typescript, this feels like a response from a decade ago, when Python was still mired in the 2 vs 3 problem.
What's happened to the popularity of all of these languages since 2010? Outside of JS/TS, absolutely nothing. If anything, they've lost mindshare.
JavaScript
Typescript
OCaml
Haskell
F#