`context` is a number array that gets returned with each /generate request and can be returned to the server with the next request. There's not much more documentation than what's there in that link, but as I understand it the array is a JSON representation of the current state of inference at the time that Ollama returned. So when you send the context back up with a second request, Ollama can just load that array into memory and then start inference with the new prompt, rather than having to re-encode the whole conversation up to that point.

I tested it, and it definitely works like a history - eg you can feed it to a model and ask it a question about what you just talked about, and it works.

Does Ollama essentially just convert the conversation so far into a binary representation, then convert it back to text and feed it back to the model ahead of your newest query? Or is it doing something more involved?

I wish it was better documented, have a ton of questions about how it functions in practice that I’ll end up trying to reverse engineer.

Like, what’s the lifetime of this context? If I load a new model into memory then reload the original, is that context valid? Is it valid if the computer restarts? Is it valid if the model gets updated?

Not an LLM expert but based on your explanation that it’s related to embedding (and another comment that Ollama loads it directly into memory) then I’m guessing that the model updating its weights definitely invalidates the context. Not so sure about the other options, like unloading/reloading the same model.

Any particular resources you’d recommend to learn more about this?

So if I’m understanding it correctly, there’s one consistent way that Ollama will vectorize a set of text. Perhaps there are various ways one could, but Ollama chooses one.

What about multimodality? Ie taking a context from a prompt to llava to identify an image, with further questions about the contents of that image? Any non-llava model would definitely hallucinate, but would llava?