News for hackers. Stuff that matters.

A comment on Gradient AI Releases 1M Context Llama 8B. Parent comment

Re:Gradient AI Releases 1M Context Llama 8B by throwaway4aday on Monday April 29, 2024 @06:14PM (#40204744)
can you select a context length that fits in your GPU though? I suppose even a 128k model would be more than enough for almost everyone running these models on their own hardware.
[ Parent | on HN ]
Re:Gradient AI Releases 1M Context Llama 8B by wkat4242 on Monday April 29, 2024 @09:21PM (#40206273)
No you can't right now. Hopefully they will add this to ollama.
[ Parent | on HN ]
Re:Gradient AI Releases 1M Context Llama 8B by ml2 on Tuesday April 30, 2024 @07:33AM (#40209846)
256k (actually 262k) is also up on HF: https://huggingface.co/gradientai/Llama-3-8B-Instruct-262k
[ Parent | on HN ]