News for hackers. Stuff that matters.
Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by rescripting on Thursday May 11, 2023 @02:42PM (#35906438)
A simple example I ran in to was I asked ChatGPT to generate me story in madlibs format for my 4 year old daughter. They're in the format "The young _____ went to the ______, ...", and she fills in the blanks with silly nouns/adjectives.

As she kept asking for more, I prompted "great, do another one" and eventually my original instruction fell out of the context window. It continued to generate a children's story, but with no more blanks.

Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by com2kid on Thursday May 11, 2023 @03:02PM (#35906782)
This is actually a different issue, largely a UI one, although one I wish ChatGPT would fix it.

There is no good way to tell it "this isn't a conversation, just repeat the answer to the initial prompt again".

The solution is to just re-paste the initial prompt in each time, but still it isn't ideal. There isn't a good way to tell chatgpt "you can throw away all the context after the initial prompt and up until now".

Of course the entire point of ChatGPT is that it maintains a conversation thread, so I get why they don't fix up this edge case.

My problem is more of, I give ChatGPT some complicated instructions, and it'll start forgetting the early on instructions long before any token limit is reached.

So for example, if early on I ask for certain tokens to be returned in parens, well my initial prompt is too long, it'll forget the parens thing and start returning tokens without the surrounding (), which then breaks my parser!

Re:Anthropic - Introducing 100K Token Context Windows, Aroun... by orost on Thursday May 11, 2023 @05:40PM (#35908753)
Almost every UI for LLMs I've seen has a way to specify an initial prompt that never goes out of context, it's strange that it's not a feature in ChatGPT.