Re:After 118 days, GPT Unicorn has produced a unicorn
by noduerme on Thursday August 10, 2023 @09:03AM
(#37075639)
If by batching you mean bad code that fails to sort or relies on hardware to best-guess how things sort, then sure, that's called a bug. Also, "we're pretty sure" is rather self-important while also admitting total, abject failure to produce a deterministic result. You shouldn't blame yourself. A lot of people had the same feeling after staking their life on the revolutionary properties of NFTs.
[ Parent
| on HN ]
Re:After 118 days, GPT Unicorn has produced a unicorn
by n2d4 on Thursday August 10, 2023 @09:31AM
(#37075938)
Your incorrect assumption here is that determinism comes with no tradeoffs. There are a few outsider analyses on the topic, for example, this one on sparse MoEs [1]. If OpenAI uses sparse MoE as described, then determinism would be possible but inefficient.
Even if it's not sparse MoE, chances are high that the non-determinism is introduced somewhere purely as a performance optimization. The article speculates that OpenAI knows this well and hides it to protect the model internals.
[1] https://152334h.github.io/blog/non-determinism-in-gpt-4/