News for hackers. Stuff that matters.
Re:Dola Decoding by Contrasting Layers Improves Factuality i... by dinobones on Wednesday July 10, 2024 @07:57PM (#40932636)
Why is this surprising?

It makes sense that “facts” exist in earlier layers, then these become more abstract as you become deeper.

This reminds me of residual connections from CNNs and vision.

Re:Dola Decoding by Contrasting Layers Improves Factuality i... by snthpy on Thursday July 11, 2024 @02:39AM (#40934156)
Interesting. I hadn't considered that before but makes sense.