Re:Run llama3 locally with 1M token context
by ukuina on Tuesday April 30, 2024 @07:20PM
(#40217669)
Tested this on Apple Silicon at 128k context, very poor results with random character hallucinations.
[ Parent
| on HN ]
Re:Run llama3 locally with 1M token context
by growfeather on Wednesday May 01, 2024 @12:47AM
(#40219602)
Thanks for testing it. I wonder if there’s a reasonable line? 64k? How much RAM did the 128k context require? (Maybe future models might be better at that level)