Re:Scalable MatMul-Free Language Modeling
by naasking on Sunday June 09, 2024 @02:05PM
(#40626272)
I feel like all of these transformer reductions to binary or ternary bits are basically constructing an implicit decision tree, where any stage of the process is basically answering a question with yes/no/I don't know answers, where "I don't know" basically invokes a continuation for further processing with more context.
[ Parent
| on HN ]
There are no comments on this comment yet.