News for hackers. Stuff that matters.
Re:AI leaderboards are no longer useful. It's time to switch... by exe34 on Tuesday April 30, 2024 @01:36PM (#40213630)
They can be made to be deterministic on CPU, but not on GPU (unless you want to give up on the speedup). With floating points, things like addition are not associative: a + (b + c) is not the same as (a + b) + c. So on CPU, you can make sure the order is always the same and the result is deterministic. On GPU, the order is not guaranteed, and thus the output is not deterministic.

This is because of the

There are no comments on this comment yet.