News for hackers. Stuff that matters.
Re:Non-determinism in GPT-4 is caused by Sparse MoE by xiphias2 on Saturday August 05, 2023 @04:03AM (#37010002)
Using atomics is easier than warp operations (using warp shuffle for example), but warp shuffle is quite fast.

I guess if determinism is so important implementations can be changed, it is just maybe not that high priority.

There are no comments on this comment yet.