Re:Non-determinism in GPT-4 is caused by Sparse MoE
by DeathArrow on Saturday August 05, 2023 @02:43AM
(#37009590)
If the hardware is deterministic, so are the results. You can't generate random numbers purely in software with deterministic hardware.
[ Parent
| on HN ]
Re:Non-determinism in GPT-4 is caused by Sparse MoE
by WithinReason on Saturday August 05, 2023 @02:58AM
(#37009663)
The behaviour of atomic operations is definitely not deterministic. E.g. if you have a lot of atomic adds, every time you run the code you'll get a different result without a random number generator.