Neural Networks Are Already Clean

Re:Neural Networks Are Already Clean by NaiveEmpiricist on Thursday October 15, 2026 @05:23PM (#50110478)

I'd be interested to see where this goes, but it seems to rely on LLMs being mostly or entirely compositional (i.e. ultimately, they can be taken apart to be understood). But if LLMs have notable emergent properties - which seems likely - then those parts of the LLM will be effectively off-limits to this sort of interpretation.

And emergence doesn't need to involve much of the LLM's weights:

> In preliminary audits of Qwen3-32B, we estimate that roughly 90% of neuron poles have one main job, often with small related offshoots; another ~7% have two related jobs; and only ~2-4% are not yet understood

If those ~2-4% are the central hubs of emergent cognition, then understanding 96%-98% of the weights may be less valuable than the author thinks.

[ Parent | on HN ]

Re:Neural Networks Are Already Clean by emp17344 on Thursday October 15, 2026 @09:37PM (#50112691)

Emergence within LLMs is pretty much unproven, and this seems like pretty strong evidence that nothing is really “emerging” here.

[ Parent | on HN ]

Re:Neural Networks Are Already Clean by NaiveEmpiricist on Friday October 16, 2026 @10:23AM (#50118732)

"Unproven" is in the eye of the beholder, but to me it certainly looks like the sudden take-off in arithmetic ability (to name one example) at approx 10^22 training FLOPs across all LLMs is emergent. (ref. https://hai.stanford.edu/news/examining-emergent-abilities-l...)

[ Parent | on HN ]

Re:Neural Networks Are Already Clean by hargup on Friday October 16, 2026 @12:02AM (#50113932)

“Emergence” from LLMs is very similar to emergence from scaling transistors.

When you have only hundreds of them, you can barely add numbers. But the same fundamental technology scaled to millions, billions and then trillions, transistors can do things which will look magic to previous generations.

[ Parent | on HN ]