Re:Building LLMs from the Ground Up: A 3-Hour Coding Workshop
by nerdponx on Sunday September 01, 2024 @12:37PM
(#41418234)
Yes, but "what the thing is actually doing" is different depending on what your perspective is on what "the thing" and what "actually" consists of.
If you are interested in how the model works conceptually, how training works, how it represents text semantically, etc., then I maintain that computational details are an irrelevant distraction, not an essential foundation.
How about another analogy? Is SICP not a good foundation for learning about language design because it uses Scheme and not assembly or C?
People are looking at the ground up for a clear picture of what the thing is actually doing, so masking the important part of what is actually happening, then calling it “ground up” is disingenuous.