Re:Llama2.c: inference llama 2 in one file of pure C
by immibis on Monday July 24, 2023 @02:45AM
(#36843876)
Sounds like a good avenue to research, but you probably want to generate more than 2 tokens ahead. Try 20 tokens, but I suppose you don't want N^20 executions of the LLM, but more like a representative sampling of say 200 combinations of the next 20 tokens. I don't know how you'd do that.
[ Parent
| on HN ]
There are no comments on this comment yet.