I'm starting to explore this for Time-Series analysis -- still at the beginning as I mostly used C++ before. Torch.ch (so old now!) used LuaJIT which has excellent FFI.

What is the realm of the interop problem there? That one wants to assemble data in Go and then feed it to another API? Or even at a lower level, getting data to a GPU/TPU requires C interop so it's more about the memory sharing?

Those libraries come from Google, so hopefully they can help battle these issues in the nether realms.

Can you elaborate on this or point me to some more information? I am primarily a Go dev that has been integrating AI/ML into a personal project. Like many other apps, I wrap llama.cpp and various other popular ML projects.

Recently I have been getting the itch to go lower level and see if it would be feasible to write an LLM backend in pure Go, or a Stable Diffusion backend. Just for curiosity and learning.

But if this is a dead-end I would like to know from someone more experienced than I. Thanks for any advice you can offer in advance.

Do you know of any resources with details on using llama.cpp with Go?

Why not elixir then? Good c interop, functional, and good performance.

Erlang/Elixir are very slow for most computationally expensive tasks.

Isn’t python the same? Most of the work is done by c++ or rust libs with python as glue.

Python is the same but there's not nearly enough incentive to move to another slow language. A lot of people that are interested in using something other than Python want a language that doesn't require everything to be written in C, C++ or Fortran and used via bindings. Having most of the ecosystem written in one expressive and relatively fast language has huge benefits.

At my last company I built a Go tensorflow inference server.. it worked well