News for hackers. Stuff that matters.
Re:MistralLite by Amazon Web Services by simonw on Wednesday November 01, 2023 @01:51PM (#38102070)
Aaah I see - so it's actually more like:

    <|prompter|>... prompt goes here</s>
    <|assistant|> ... assistant response will go here, maybe followed by </s>
Still don't understand why I got a rogue </s> in the reply when I ran a prompt through it (but it continued after that tag): https://news.ycombinator.com/item?id=38101766

And is my guess about </s> at the end of the assistant thing correct? If so then a conversation might look like this:

    <|prompter|>hi</s>
    <|assistant|>Hi to you too. What do you want to talk about?</s>
    <|prompter|>Tell me facts about snakes</s>
    <|assistant|>
But the documentation on these models is so thin! Do I need that </s> at the end of the assistant things or not?
Re:MistralLite by Amazon Web Services by brucethemoose2 on Wednesday November 01, 2023 @02:09PM (#38102348)
> But the documentation on these models is so thin!

Welcome to LLM research land, lol. Mpstly there is no documentation, no reproducability, just random hit-and-run releases with some claimed metrics and if you're lucky a model card with the trained prompt.

> Do I need that </s> at the end of the assistant things or not?

...I dunno. I didn't add it in my prompt template, and you can see that the model itself it outputting that token in the responses. Maybe ask the trainers on the HF page.