Re:GPT-3.5 crashes when it thinks about useRalativeImagePath...
by shawntan on Monday January 22, 2024 @02:46AM
(#39087305)
Exactly this. The tokens generated should always be valid, unless some post-processing layer between the model's output and the user interface detects for some keywords which it would prefer to filter out. In which case I suppose there is another commonly seen error message that appears?
[ Parent
| on HN ]
Re:GPT-3.5 crashes when it thinks about useRalativeImagePath...
by SirSegWit on Monday January 22, 2024 @03:00AM
(#39087364)
Not really, right? There are a ton of special tokens, like start of sequence etc., so what happens if there are two start of sequences predicted? It's a valid token but cannot really be turned into something sensible, so it throws an error when converting tokens to plain text?