News for hackers. Stuff that matters.

A comment on DeepSeek won the best paper award at ACL 2025.

Re:DeepSeek won the best paper award at ACL 2025 by sabakhoj on Friday August 01, 2025 @08:01PM (#44763752)
> Despite being sparse, NSA surpasses Full Attention baseline on average across general benchmarks, long-context tasks, and reasoning evaluation.

Isn't it very notable that the latency improvement didn't have a performance loss? I'm not super familiar with all the technical aspects, but that seems like it should be one of the main focuses of the paper.

[ Parent | on HN ]
Re:DeepSeek won the best paper award at ACL 2025 by ethan_smith on Saturday August 02, 2025 @03:39AM (#44765561)
The performance maintenance (or even improvement) isn't surprising - sparse attention can reduce noise by focusing only on relevant tokens. Traditional full attention dilutes focus by attending to everything equally, while NSA's pruning approach mimics how humans selectively process information.
[ Parent | on HN ]
Re:DeepSeek won the best paper award at ACL 2025 by laughingcurve on Saturday August 02, 2025 @07:28AM (#44766720)
Yes that’s what makes it so interesting and novel you nailed it
[ Parent | on HN ]