News for hackers. Stuff that matters.

A comment on Localllm lets you develop gen AI apps on local CPUs. Parent comment

Re:Localllm lets you develop gen AI apps on local CPUs by andy99 on Wednesday February 07, 2024 @06:10PM (#39295587)
Oh it's a wrapper? I just skimmed and thought they reinvented llama.cpp for some reason, which would have been mildly interesting.
[ Parent | on HN ]
Re:Localllm lets you develop gen AI apps on local CPUs by williamstein on Wednesday February 07, 2024 @06:13PM (#39295610)
(edit: this is completely irrelevant remark. sorry) It's (1) written in Python, and (2) ollama appears nowhere in their Docker repo [1], and (3) it gets the models only from huggingface instead of the ollama repos. Thus maybe it is NOT a wrapper of ollama?

[1] https://github.com/googlecloudplatform/localllm

[ Parent | on HN ]
Re:Localllm lets you develop gen AI apps on local CPUs by andy99 on Wednesday February 07, 2024 @06:23PM (#39295685)
https://github.com/GoogleCloudPlatform/localllm/blob/main/ll...

I see llama.cpp in there, but I didn't go deeper than that. Ollama is not the same thing.

[ Parent | on HN ]