Aaah I see - so it's actually more like:
<|prompter|>... prompt goes here</s>
<|assistant|> ... assistant response will go here, maybe followed by </s>
Still don't understand why I got a rogue </s> in the reply when I ran a prompt through it (but it continued after that tag): https://news.ycombinator.com/item?id=38101766And is my guess about </s> at the end of the assistant thing correct? If so then a conversation might look like this:
<|prompter|>hi</s>
<|assistant|>Hi to you too. What do you want to talk about?</s>
<|prompter|>Tell me facts about snakes</s>
<|assistant|>
But the documentation on these models is so thin! Do I need that </s> at the end of the assistant things or not?
Re:MistralLite by Amazon Web Services
by brucethemoose2 on Wednesday November 01, 2023 @02:09PM
(#38102348)
> But the documentation on these models is so thin!
Welcome to LLM research land, lol. Mpstly there is no documentation, no reproducability, just random hit-and-run releases with some claimed metrics and if you're lucky a model card with the trained prompt.
> Do I need that </s> at the end of the assistant things or not?
...I dunno. I didn't add it in my prompt template, and you can see that the model itself it outputting that token in the responses. Maybe ask the trainers on the HF page.
<|prompter|> is the human message (including the instructon/contexts from my tests) and <|assistant|> is the bot's response.
There are llm wrappers that will save chat histories and reformat them appropriately for different LLMs. For instance, see TUI and its instruction template presets: https://github.com/oobabooga/text-generation-webui/tree/main...