News for hackers. Stuff that matters.
Re:Hacking Around ChatGPT’s Character Limits with the Code I... by BoiledCabbage on Sunday May 28, 2023 @01:53PM (#36106211)
I think you misunderstood the token limit. LLMs don't block your buffer, they simply take the final n tokens of all of the input you've shared. Plenty of users have seen this.

It will still function, but anything you previously shared and referenced above it will lose context on. And if you ask it about that earlier content, it will do its best to hallucinate a reasonable answer of what might have been in your buffer before the cutoff.

Separately you may have found a physical hard limit with a bug that crashes the system, but that's not what's meant by a token limit in LLMs. It's a limitation of the architecture itself of any LLM.

There are no comments on this comment yet.