News for hackers. Stuff that matters.
Re:Quadrillion-parameter transformers and how to train them by ben_w on Sunday November 08, 2026 @10:33AM (#50414129)
Our entire genome would fit on a CD, and much of that is used for metabolism and overall body morphology, not specifically brain structure; even though it almost certainly gives us a whole bunch of stuff (e.g. enough to recognise smiles and thereby allow them to form a positive reward we can learn from, and some approximation of how to sense a potential reproductive partner), it is absolutely not enough to give us a universal language nor intuition about any tool more advanced than a sharp rock and possibly the wheel and fire.

On the other hand, even such basics may well make a huge difference.

Re:Quadrillion-parameter transformers and how to train them by noduerme on Sunday November 08, 2026 @10:53AM (#50414296)
I think it's the training set.

Pretty much every organism with a brain has a sophisticated survival reaction to physical contact with any part of its body, because its brain is trained on nerve signals. Everything with eyes or sonar learns spacial awareness of objects and their proximity, velocity, directoion of motion. I think the clue is that biological brains are trained on a physical world with very complex rules and physics as well as reactions that arise from chemistry.

Words and images, audio and video, even virtual worlds with soft-body physics, are just flat projected shadows of the complexity of the actual world.

Re:Quadrillion-parameter transformers and how to train them by ben_w on Sunday November 08, 2026 @11:50AM (#50414864)
> Words and images, audio and video, even virtual worlds with soft-body physics, are just flat projected shadows of the complexity of the actual world.

While true, so too are our senses: https://en.wikipedia.org/wiki/Aristotelian_physics

Re:Quadrillion-parameter transformers and how to train them by Teever on Sunday November 08, 2026 @01:17PM (#50415632)
One time when I was a teenager I was outside with my house cat and she was doing normal housecat stuff like rolling around on the ground or sniffing stuff until suddenly for no apparent reason she went into a prone position with her limbs spread out and then bolted around in an odd zig-zag pattern and made her way to the corner of the garage overhang and jumped into the little opening that lead to the attic in the garage

I wondered wtf was going on so I looked around and realized that some sort of bird of prey was circling overhead that triggered that reaction and behaviour. It got me thinking. Somewhere inside of her brain and therefore her genome was the equivalent of a pixelated black and white image of the outline of a bird of prey, and the algorithm that can generate that zig-zag pattern in muscle firings and a whole overall heuristic that can apply these patterns and others into a coherent strategy that manifested as the fight or flight behaviour that I saw.

No one taught her this as she never really spent any time with her parents or her litter as she was adopted quite young, and spent most of her time doing house cat things so it wasn't something that she picked up from her environment, so it seemed pretty innate to me.

It was very influential to me as a teen to ponder that if that bird of prey's genome encodes the full shape and structure and behaviour of it then in some fractal sense there's a low fidelity copy of the shape of that bird and also implicity the bird's behaviour through a counter prey behaviour encoded in the cat's genome.