News for hackers. Stuff that matters.

A comment on Comparing 60 LLMs with a set of 20 prompts. Parent comment

Re:Comparing 60 LLMs with a set of 20 prompts by detente18 on Saturday September 09, 2023 @11:44AM (#37446684)
Here's the template I'm using - https://github.com/BerriAI/litellm/blob/5ca8b23e22139a4f49bd...

Anything I'm doing incorrectly?

[ Parent | on HN ]
Re:Comparing 60 LLMs with a set of 20 prompts by YetAnotherNick on Saturday September 09, 2023 @09:48PM (#37452148)
The format is really weird for chat models. In your code post message is `<</SYS>>\n [/INST]` which is wrong. Instead the overall format is `[INST] <<SYS>>\n{system}\n<</SYS>>\n\n{instruction} [/INST]`

So both the system message and first instruction are in same [INST] block.

[ Parent | on HN ]