News for hackers. Stuff that matters.

A comment on Backprop Alternative: Augmented Lagrangian Predictive Coding. Parent comment

Re:Backprop Alternative: Augmented Lagrangian Predictive Coding by verdverm on Tuesday September 15, 2026 @03:58AM (#49709221)
I'm not convinced information theory is the right avenue for such non-deterministic, environmentally affected, open ended systems. Protein synthesis is defined by far more than the DNA, which is always being processed, most of which is junk and being recycled. The layers in between are more than interpretation of the prior layers, the data grows at each level to incorporate more sources under your information theoretic formulation.

I was never paraphrasing you, but thank you for attempting rhetorical antics?

[ Parent | on HN ]
Re:Backprop Alternative: Augmented Lagrangian Predictive Coding by qarl on Tuesday September 15, 2026 @07:00AM (#49710577)
> I'm not convinced information theory...

I'm not convinced my molecules follow the laws of thermodynamics.

[ Parent | on HN ]
Re:Backprop Alternative: Augmented Lagrangian Predictive Coding by Xmd5a on Tuesday September 15, 2026 @12:27PM (#49714958)
Information theory is the right avenue but it's too often used to imply things it says nothing about. From (1)

1. A determinist transformation can't create more information than its inputs

you can't derive (2)

2. Later stages don't contain any new information.

without restraining your model to a monotonic chain of transformations that can't tap into the information content of the environment.

[ Parent | on HN ]