News for hackers. Stuff that matters.
Re:Understand how transformers work by demystifying all the ... by thrtythreeforty on Thursday January 04, 2024 @01:23AM (#38863749)
Can you link a resource that is able to adequately explain why they're called Key, Query, and Value? Every explanation I've read eventually handwaved this. It feels like understanding why they're named that is key (heh) to understanding the concept, rather than just blindly implementing matmul.
Re:Understand how transformers work by demystifying all the ... by pests on Thursday January 04, 2024 @04:01AM (#38864739)
https://stats.stackexchange.com/questions/421935/what-exactl...

It mentions it comes from the original Attention Is All You Need paper and goes on into more detail.

It seems to be named exactly as you would expect. Key/Value as in KV store, with Query being the term being retrived.