News for hackers. Stuff that matters.
Re:The Secret Sauce behind 100K context window in LLMs: all ... by avereveard on Sunday June 18, 2023 @04:23AM (#36378316)
I use a sandwitch approach: system message contains instruction, then I pass it a user message with the context, and last a agent message with "I will now process this data according to the instruction for (short summary of system message) as (format):"

then I ask to generate. it's very powerful, as it removes the preamble and other chitcat from the response, and empower the system message over what's in the user message.

example: https://i.imgur.com/7fF0CZm.png?maxwidth=123456789&fidelity=... here the first agent message is the one conditioning the answer beginning, and I only generate the second agent.

(sorry mobile user imgur may return a low res unreadable image idk what's the alternative in 2023)

Re:The Secret Sauce behind 100K context window in LLMs: all ... by furyofantares on Sunday June 18, 2023 @12:56PM (#36381985)
Adding an agent message at the end is an excellent idea.