News for hackers. Stuff that matters.
Re:Google scrambles to manually remove weird AI answers in s... by Terr_ on Saturday May 25, 2024 @05:00PM (#40477832)
I think what you're describing is that training/execution effects aren't predictable.

It is still "deterministic" in that training on exactly the same data and asking exactly the same questions should (unless someone manually adds randomness) lead to the same results.

Another example of the distinction might be a pseudo-random number generator: For any given seed, it is entirely deterministic, while at the same time being very deliberately hard to predict without actually running it to see what happens.

Re:Google scrambles to manually remove weird AI answers in s... by dleeftink on Saturday May 25, 2024 @11:52PM (#40479676)
True in the ideal case, but taken together (e.g. corpus retraining, temperature settings, slight input changes, initial descent parameters) unpredictability and indeterminism become difficult to distinguish. Especially in the distributed training case, training data may be propagated to different nodes in different order (e.g. when leaving it to a query optimiser), which makes any large-scale training operation difficult to reproduce exactly.