News for hackers. Stuff that matters.
Re:GPT-3.5 crashes when it thinks about useRalativeImagePath... by npsomaratna on Sunday January 21, 2024 @11:51PM (#39086352)
Correct me if I'm wrong—but we don't know if GPT-4 uses the same tokenizer as GPT-3.5, right?
Re:GPT-3.5 crashes when it thinks about useRalativeImagePath... by minimaxir on Sunday January 21, 2024 @11:55PM (#39086372)
OpenAI's web tokenizer demo confirms it: https://platform.openai.com/tokenizer