News for hackers. Stuff that matters.
Re:MistralLite by Amazon Web Services by brucethemoose2 on Wednesday November 01, 2023 @12:07PM (#38100295)
Yeah, this is a great finetune that kinda flew under the radar. It will indeed take 32K context without hacky context extension (and theoretically should work OK stretched to 48K).

It still behaves more like an good unaligned LlamaV2 13B finetune.

And it loves to summarize and retrieve facts. In fact, when I give it a huge context and instruct it to storytell, deduct or whatever, sometimes it will summarize the context instead... But the summary is excellent.

Re:MistralLite by Amazon Web Services by hfjjbf on Wednesday November 01, 2023 @12:46PM (#38101008)
In my experience llama2 13b is fairly worthless in comparison to 70b.

I think L70b is the mark to beat, I’m hopeful mistral will take the crown with a smaller model.

Re:MistralLite by Amazon Web Services by brucethemoose2 on Wednesday November 01, 2023 @12:55PM (#38101147)
Mistral 7B is way better than LlamaV2 13B chat.

But some of the 3rd party 13B finetunes/merges are also much better that Meta's chat model.