News for hackers. Stuff that matters.
Re:Scalable MatMul-Free Language Modeling by naasking on Sunday June 09, 2024 @02:05PM (#40626272)
I feel like all of these transformer reductions to binary or ternary bits are basically constructing an implicit decision tree, where any stage of the process is basically answering a question with yes/no/I don't know answers, where "I don't know" basically invokes a continuation for further processing with more context.

There are no comments on this comment yet.