News for hackers. Stuff that matters.

A comment on Open source LLM with 32k Context Length.

Re:Open source LLM with 32k Context Length by vekker on Thursday August 24, 2023 @05:13AM (#37246653)
It's probably too new for anyone to have integrated this into text-generation-webui / Gradio? I've been looking for a large context LLM (self-hosted or not) for a project, and as a European I unfortunately don't have access to Anthropic's Claude API yet.
[ Parent | on HN ]
Re:Open source LLM with 32k Context Length by qeternity on Thursday August 24, 2023 @05:26AM (#37246746)
It's just Llama 2 w/ rotary encoding fine tuned to 32k. It should work fine.
[ Parent | on HN ]
Re:Open source LLM with 32k Context Length by syntaxing on Thursday August 24, 2023 @07:22AM (#37247559)
How high context do you need? There’s a couple 16K models out there now. Some people have their own 32K ones too but the quality vary. It’s worth trying them on huggingface. The easiest way is to track TheBloke’s work to see any new models that come out.
[ Parent | on HN ]