News for hackers. Stuff that matters.
Re:Apple: Embarrassingly Simple Self-Distillation Improves C... by joshuaisaact on Sunday April 05, 2026 @03:50AM (#47647147)
This was a really interesting paper but there's a massive gap in what they didn't try, which is inference-time temperature changes based on the fork/lock distinction.

Maybe I'll try that myself, because it feels like it could be a great source of improvements. It would be really useful to see adaptive per-token sampling as an additional decode-only baseline.

Re:Apple: Embarrassingly Simple Self-Distillation Improves C... by grumbelbart on Sunday April 05, 2026 @06:34AM (#47648010)
Is this some kind of calibration then? I'd expect that the probabilities automatically adjust during training, such that in "lock" mode, for example, syntax-breaking tokens have a very low probability and would not be picked even wich higher temperature.
Re:Apple: Embarrassingly Simple Self-Distillation Improves C... by tatrions on Sunday April 05, 2026 @02:54PM (#47652636)
Token-level entropy is a pretty clean proxy for detecting fork vs lock positions at inference time. Low entropy = lock (decode greedily), high entropy = fork (sample with more temperature). Speculative decoding already exploits something similar where the small draft model handles the predictable tokens and the big model kicks in at the uncertain ones. Combining that with this paper's fork/lock framing could get you adaptive temperature basically for free during inference.