News for hackers. Stuff that matters.
Re:Deepseek R1-0528 by alfiedotwtf on Thursday May 29, 2025 @04:36AM (#44124130)
So is there no “introduce randomness” at some step afterwards? If not, I would guess these models would be getting stuck in a local maxima
Re:Deepseek R1-0528 by addaon on Thursday May 29, 2025 @12:05PM (#44127406)
> If not, I would guess these models would be getting stuck in a local maxima

It sounds like you're referring to something like simulated annealing. Using that as an example, the fundamental requirement is to introduce arbitrary, uncorrelated steps -- there's no requirement that the steps be random, and the only potential advantage of using a random source is that it provides independence (lack of correlation) inherently; but in exchange, it makes testing and reproduction much harder. Basically every use of simulated annealing or similar I've run into uses pseudorandom numbers for this reason.