Re:Quadrillion-parameter transformers and how to train them
by Teever on Sunday November 08, 2026 @01:17PM
(#50415632)
One time when I was a teenager I was outside with my house cat and she was doing normal housecat stuff like rolling around on the ground or sniffing stuff until suddenly for no apparent reason she went into a prone position with her limbs spread out and then bolted around in an odd zig-zag pattern and made her way to the corner of the garage overhang and jumped into the little opening that lead to the attic in the garage
I wondered wtf was going on so I looked around and realized that some sort of bird of prey was circling overhead that triggered that reaction and behaviour. It got me thinking. Somewhere inside of her brain and therefore her genome was the equivalent of a pixelated black and white image of the outline of a bird of prey, and the algorithm that can generate that zig-zag pattern in muscle firings and a whole overall heuristic that can apply these patterns and others into a coherent strategy that manifested as the fight or flight behaviour that I saw.
No one taught her this as she never really spent any time with her parents or her litter as she was adopted quite young, and spent most of her time doing house cat things so it wasn't something that she picked up from her environment, so it seemed pretty innate to me.
It was very influential to me as a teen to ponder that if that bird of prey's genome encodes the full shape and structure and behaviour of it then in some fractal sense there's a low fidelity copy of the shape of that bird and also implicity the bird's behaviour through a counter prey behaviour encoded in the cat's genome.
I wondered wtf was going on so I looked around and realized that some sort of bird of prey was circling overhead that triggered that reaction and behaviour. It got me thinking. Somewhere inside of her brain and therefore her genome was the equivalent of a pixelated black and white image of the outline of a bird of prey, and the algorithm that can generate that zig-zag pattern in muscle firings and a whole overall heuristic that can apply these patterns and others into a coherent strategy that manifested as the fight or flight behaviour that I saw.
No one taught her this as she never really spent any time with her parents or her litter as she was adopted quite young, and spent most of her time doing house cat things so it wasn't something that she picked up from her environment, so it seemed pretty innate to me.
It was very influential to me as a teen to ponder that if that bird of prey's genome encodes the full shape and structure and behaviour of it then in some fractal sense there's a low fidelity copy of the shape of that bird and also implicity the bird's behaviour through a counter prey behaviour encoded in the cat's genome.