News for hackers. Stuff that matters.

A comment on Show HN: Kalosm an embeddable framework for pre-trained models in Rust.

Re:Show HN: Kalosm an embeddable framework for pre-trained m... by dvt on Wednesday February 28, 2024 @01:35PM (#39541959)
Starting work on a product where I'll need RAG + some language model (maybe llama) and Kalosm seems interesting. However, I'd like to package the model with the app. I don't really like the new trend of on-demand downloading the model via a library in some random cache folder on the user's computer (which services like Huggingface have popularized).

Is there any non-hacky way of doing this?

[ Parent | on HN ]
Re:Show HN: Kalosm an embeddable framework for pre-trained m... by Evan-Almloff on Wednesday February 28, 2024 @01:50PM (#39542117)
Yes, you can set the source to any local file instead of a huggingface model. Here is a example: https://gist.github.com/ealmloff/3398d172180fa783f043b4a2696...
[ Parent | on HN ]