News for hackers. Stuff that matters.

A comment on Gemini Flash. Parent comment

Re:Gemini Flash by falcor84 on Tuesday May 14, 2024 @05:11PM (#40360210)
I wonder if we could implement the equivalent of a JIT compilation, whereby context sequences which get repeatedly reused would be used for an online fine-tuning.
[ Parent | on HN ]
Re:Gemini Flash by HumanOstrich on Tuesday May 14, 2024 @10:10PM (#40362360)
This is like some developer terms mixed into a word salad that might sound good to an amateur but makes no sense in the current context.
[ Parent | on HN ]
Re:Gemini Flash by nostrebored on Tuesday May 14, 2024 @10:49PM (#40362586)
They are asking if you can take the context being passed per interaction and train it into a session in real time (via an online algorithm).

Essentially bake the context passed in to the attention layer so that you can pass only the relevant chat context.

Your post wasn’t a particularly charitable interpretation.

[ Parent | on HN ]
Re:Gemini Flash by sp332 on Tuesday May 14, 2024 @10:45PM (#40362561)
No, but you can just cache the state after processing the prompt. https://github.com/ggerganov/llama.cpp/tree/master/examples/...
[ Parent | on HN ]