News for hackers. Stuff that matters.
Re:Things I wish I knew before building a GPT agent for log ... by doublesundae on Tuesday June 06, 2023 @12:35PM (#36215393)
Great work and thanks for sharing. Did you hit token limits? Are you storing message/ chat histories between sessions
Re:Things I wish I knew before building a GPT agent for log ... by itsderek23 on Tuesday June 06, 2023 @12:41PM (#36215494)
Thanks!

> Did you hit token limits?

While i used TikToken to limit the message history (and keep below the token limit), generally I found that I didn't get better completions by putting a lot of data into the context. Usually the completions got more confusing. I put a limited amount of info into the context and have generally stayed below the token limit.

> Are you storing message/ chat histories between sessions

Right now, yes. It's pretty important to store everything (each request / response) to debug issues with prompt, context, and the agent call loop.