Re:Show HN: Ollama – Docker for LLMs
by zenlikethat on Thursday July 20, 2023 @12:28PM
(#36802913)
I've been playing with this lately and it's been loads of fun (on OSX in particular, on Windows/WSL I don't think it's rigged up with GPTQ yet).
Of note is the experimental "Modelfile" that allows you to ship around an AI character or "stack" like a Docker image. I can really see the future around this shaping up to be really exciting, since I'll be able to hand you something much more replicable in terms of model, prompt, etc, and once we are able to chain calls and bundle embeddings, LoRa, etc., it seems like it could be really powerful.
Of note is the experimental "Modelfile" that allows you to ship around an AI character or "stack" like a Docker image. I can really see the future around this shaping up to be really exciting, since I'll be able to hand you something much more replicable in terms of model, prompt, etc, and once we are able to chain calls and bundle embeddings, LoRa, etc., it seems like it could be really powerful.