Re:Hacking Around ChatGPT’s Character Limits with the Code I...
by tomduncalf on Sunday May 28, 2023 @05:05AM
(#36102349)
> The token limit is 100% an artificial limitation.
My understanding is the token limit is an immutable property of the neural network once it has been trained, so definitely is not an artificial limitation - unless you’re suggesting OpenAI trained the NN with a higher token count then released it with a limit on the input to only allow smaller ones? Which I guess is plausible but I’m not sure why they’d do it, as they’d still be “executing” the same NN for every input so wouldn’t save any compute.
My understanding is the token limit is an immutable property of the neural network once it has been trained, so definitely is not an artificial limitation - unless you’re suggesting OpenAI trained the NN with a higher token count then released it with a limit on the input to only allow smaller ones? Which I guess is plausible but I’m not sure why they’d do it, as they’d still be “executing” the same NN for every input so wouldn’t save any compute.