Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Unlimiformer: Long-Range Transformers with Unlimited Length Input. Parent comment
1) Normalize input (batch norm, 2015)
2) Competitive dynamics / lateral inhibition (softmax in attention layers, 2017)
3) Cluster best matching activation vectors (top-k keys, 2023)
There are no comments on this comment yet.
1) Normalize input (batch norm, 2015)
2) Competitive dynamics / lateral inhibition (softmax in attention layers, 2017)
3) Cluster best matching activation vectors (top-k keys, 2023)