News for hackers. Stuff that matters.

A comment on LLM Powered Autonomous Agents. Parent comment

Re:LLM Powered Autonomous Agents by kolinko on Tuesday June 27, 2023 @05:47AM (#36490832)
With LLAMA at least, not just past tokens, but also past states for each layer and each token. So LLama has information not just on the word "Berlin", but also on why it outputted "Berlin".
[ Parent | on HN ]
Re:LLM Powered Autonomous Agents by YetAnotherNick on Tuesday June 27, 2023 @06:18AM (#36491021)
That is called caching, which is used to save computation. But only tokens are enough to calculate next token and state could be recomputed.
[ Parent | on HN ]