News for hackers. Stuff that matters.
Re:The maths you need to start understanding LLMs by stared on Saturday September 06, 2025 @06:45AM (#45148172)
Well, in short - basic linear algebra, basic probability, analysis (functions like exp), gradient.

At some point I tried to create an introduction step-by-step, where people can interact with these concepts and see how to express it in PyTorch:

https://github.com/stared/thinking-in-tensors-writing-in-pyt...

Re:The maths you need to start understanding LLMs by MichaelRazum on Saturday September 06, 2025 @07:30AM (#45148358)
Although is it really "understanding" or just able to write down the formulas...?
Re:The maths you need to start understanding LLMs by misternintendo on Saturday September 06, 2025 @07:45AM (#45148433)
In some way it is true. Like understanding how a car works purely on physics laws.
Re:The maths you need to start understanding LLMs by stared on Saturday September 06, 2025 @08:54AM (#45148872)
Being able to use a formula is the first, and necessary, step for understanding.

Then it is able to work at different levels of abstraction and being able to find analogies. But at this point, in my understanding, "understanding" is a never-ending well.

Re:The maths you need to start understanding LLMs by MichaelRazum on Saturday September 06, 2025 @09:43AM (#45149207)
How about elliptic curve cryptography then? I just think coming with a formula is not really understanding. Actually most often the “real” formula is the end step of understanding through derivation. ML does it up side down in this regard