News for hackers. Stuff that matters.
Re:An embarrassingly simple approach to recover unlearned kn... by spencerchubb on Monday November 04, 2024 @08:49AM (#42041487)
Correct me if I'm wrong, but isn't there no effect on a floating point operation if you make the numbers more precise?
Re:An embarrassingly simple approach to recover unlearned kn... by jebarker on Monday November 04, 2024 @08:57AM (#42041555)
I don't think that's always correct when you're talking about operators in neural nets. E.g. the sin and cos in rope embeddings would get more precise, large sums like softmax would become more precise, potentially attention too due to dot products