News for hackers. Stuff that matters.
Re:OpenAI postmortem – Unexpected responses from ChatGPT by hn_throwaway_99 on Wednesday February 21, 2024 @10:10PM (#39462724)
I had the exact opposite reaction. I am in no way an AI expert (or novice for that matter), but I generally have an understanding of how tokenization works and how LLMs parse text strings into a series of tokens. Thus, I thought this paragraph was particularly well-written in a manner that explained pretty clearly what happened, but in a manner accessible to a layperson like me:

> In this case, the bug was in the step where the model chooses these numbers. Akin to being lost in translation, the model chose slightly wrong numbers, which produced word sequences that made no sense.

I liked this because when I first saw the example word salads I was so impressed by them - they look to be syntactically correct, but semantically they're gibberish. But knowing the basics of how LLMs choose the next token let me imagine some bugs where the "lookup table" if you will of word-to-token or vice versa (and I realize that may not be exactly the best analogy) was slightly offset.

Further, this sentence, "More technically, inference kernels produced incorrect results when used in certain GPU configurations." led me to understand how this could make it out into production - I'm sure ChatGPT has to be using tons of different GPUs in different configs to handle their volume, so it's understandable to me that their was a new matrix of config settings + code that made this bug show up.

I don't say any of the above to "excuse" OpenAI, but I also don't think they need any "excusing" to begin with. I don't think this was an unforgivable bug, and I appreciate them being so quick to explain what happened.

Re:OpenAI postmortem – Unexpected responses from ChatGPT by ummonk on Wednesday February 21, 2024 @11:44PM (#39463350)
No, that just explains the symptom of the bug, not the underlying bug, how it came about, and how they can prevent it from happening again.

"More technically, inference kernels produced incorrect results when used in certain GPU configurations" has zero technical detail. The only information it is providing us is that the bug only showed up in some GPU configurations.

Re:OpenAI postmortem – Unexpected responses from ChatGPT by PeterisP on Thursday February 22, 2024 @05:47AM (#39465373)
I feel that it does have relevant technical detail - as far as I understand, any inference kernel is "supposed to" give the same outputs anyways, so if it produces incorrect results in certain GPU configurations, that indicates some flaw in the GPU/microcode/driver in some new hardware they introduced.