News for hackers. Stuff that matters.
Re:Donald Knuth on work habits, problem solving, and happine... by mooneater on Monday May 23, 2022 @02:23PM (#31483014)
> trying to work the concepts out in my own mind and to anticipate what the authors are going to say before turning each page. I usually fail to guess what the next page holds, but the fact that I've tried and failed makes me more ready to understand why the authors chose the paths that they did

TIL Donald Knuth operates a bit like GPT-3 but for research paper narrative.

Re:Donald Knuth on work habits, problem solving, and happine... by ohwellhere on Monday May 23, 2022 @02:28PM (#31483096)
I see it differently. It's not prediction based on anything statistical but based on one's understanding.

I've made it a habit to ask myself what I expect the output to be for any programming operation, and why. It forces me to gain clarity into my mental model of what's happening, and it immediately highlights deficiencies in my model when it's proven wrong.

I ask the same of others when I pair program with juniors or interviewees. I find super useful all around.

Re:Donald Knuth on work habits, problem solving, and happine... by mooneater on Monday May 23, 2022 @02:35PM (#31483193)
> not prediction based on anything statistical but based on one's understanding

It is still statistical. You still want to predict a distribution over expected next token.

However, the function needed to estimate the likely next tokens are not simple max over enumerated next tokens like in a language model. It's more like a transition model in reinforcement learning.

Re:Donald Knuth on work habits, problem solving, and happine... by NateEag on Monday May 23, 2022 @08:38PM (#31486649)
> It is still statistical. You still want to predict a distribution over expected next token.

The fact that a process can be described statistically does not make the process itself statistical.

Re:Donald Knuth on work habits, problem solving, and happine... by DeathArrow on Tuesday May 24, 2022 @02:01AM (#31488516)
With the current craze of neural networks people would like to think that anything that happens in the Universe is statistical. But it's really not.
Re:Donald Knuth on work habits, problem solving, and happine... by passion__desire on Tuesday May 24, 2022 @12:25PM (#31494136)
Quantum mechanics is probablistic. But the probabilities are deterministic.