News for hackers. Stuff that matters.
Re:Meta Llama 3 by pama on Friday April 19, 2024 @12:10AM (#40083359)
I personally think that the model artifacts are simply programs with tons of constants. Many math routines have constants in their approximations and I don’t expect the source to include the full derivation for these constants all the time. I see LLMs as a same category but with (much) larger sets of parameters. What is better about the LLMs than some of the mathematical constants in complicated function approximations, is that I can go and keep training an LLM whereas the math/engineering libraries might not make it easy for me to modify them without also figuring out the details that led to those particular parameter choices.

There are no comments on this comment yet.