News for hackers. Stuff that matters.
Re:Jets: The Ruby Serverless Framework by ElectricalUnion on Friday December 08, 2023 @11:52PM (#38578712)
> Most LLM client code uses LangChain

From what I saw on several LLM projects, they usually don't use LangChain at all - if they really care about performance, they go straight to huggingface/transformers or even directly to pytorch.

If you want use PyTorch and you're dead-set into using Ruby then you can just FFI LibTorch and you should be fine; torch-rb mostly does that.

Does it work "OK"? Well, I would say for some cases potentially better that Python, but mostly because in python-land version and package management is somewhat of a unmitigated dumpster fire.

Re:Jets: The Ruby Serverless Framework by itake on Monday December 11, 2023 @06:49PM (#38607111)
LangChain is an API layer wrapping huggingface and/or pytorch. Huggingfacetransformers and pytorch solve a different set of problems than what LangChain solves.

I understand that, but most programmers want the simple `pip install` `gem install` flow to work, without needing to create their own bindings code.

torch-rb offers the barebones. You'd still need to re-implement the model from whitepapers or translate the python version.