News for hackers. Stuff that matters.
Re:Quadrillion-parameter transformers and how to train them by magicalhippo on Sunday November 08, 2026 @01:12PM (#50415598)
I recall that nature documentary showing how kangaroo fetuses climb up and into mom's pouch. They've been in a womb for the human equivalent of 7 weeks, yet immediately outside they know they must climb up, how to crawl through the fur, and how to identify when they've reached their destination.

I'm also reminded of that car AI using a neural net with a single shared weight, where the topology of the neural graph was what made the AI work.

How stuff is connected is what our DNA is about, so seems obvious the topology must encode behavior and such and it's not all learned.

Re:Quadrillion-parameter transformers and how to train them by alwa on Sunday November 08, 2026 @01:33PM (#50415747)
Is this the line of “single shared weight” work that you’re referring to?

https://weightagnostic.github.io/

Re:Quadrillion-parameter transformers and how to train them by magicalhippo on Sunday November 08, 2026 @03:14PM (#50416681)
[delayed]