News for hackers. Stuff that matters.
Re:Llama 32K Context Released by Together AI by lamuswawir on Saturday July 29, 2023 @04:01AM (#36917981)
I have seen this also, but attention is far better for gtp-4, it seems to follow a system prompt for say, outputing json, uniformly compared to gpt-3.5. I have also found that gpt-3.5 follows a system prompt to do the same for only 2 successive outputs. You have to give it that prompt with every single time. So I think increasing context windows may not make it follow the system prompt uniformly.

There are no comments on this comment yet.