News for hackers. Stuff that matters.
Re:Tensor Product Attention Is All You Need by zaptrem on Wednesday January 22, 2025 @10:30PM (#42800403)
Transformers are deep feedforward networks that happen to also have attention. Causal LMs are super memory bound during inference due to kv caching as all of those linear layers need to be loaded onto the core to transform only a single token per step.
Re:Tensor Product Attention Is All You Need by menaerus on Thursday January 23, 2025 @03:30AM (#42801966)
And I said something else?