News for hackers. Stuff that matters.

A comment on Tree Attention: Topology-Aware Decoding for Long-Context.

Re:Tree Attention: Topology-Aware Decoding for Long-Context by brrrrrm on Sunday August 11, 2024 @08:50PM (#41220320)
how does this approach differ from Nvidia's 2019 writeup on using trees to improve allreduce operations? https://developer.nvidia.com/blog/massively-scale-deep-learn...
[ Parent | on HN ]
Re:Tree Attention: Topology-Aware Decoding for Long-Context by mrfox321 on Monday August 12, 2024 @08:01AM (#41223459)
Similar, but specialized to softmax(QK)V computation
[ Parent | on HN ]