News for hackers. Stuff that matters.
Re:Underrated reasons to be thankful V by stevenhuang on Thursday November 27, 2025 @05:35PM (#46073785)
> 21. That every expression graph built from differentiable elementary functions and producing a scalar output has a gradient that can itself be written as an expression graph, and furthermore that the latter expression graph is always the same size as the first one and is easy to find, and thus that it’s possible to fit very large expression graphs to data.

> 22. That, eerily, biological life and biological intelligence does not appear to make use of that property of expression graphs.

Claim 22 is interesting. I can believe that it isn't immediately apparent because biological life is too complex (putting it mildly), but is that the extent of it?

Re:Underrated reasons to be thankful V by kragen on Thursday November 27, 2025 @08:46PM (#46074837)
We haven't found anything in nature that resembles reverse-mode automatic differentiation, either in evolution or in neuroscience.
Re:Underrated reasons to be thankful V by mountainreason on Friday November 28, 2025 @08:03PM (#46084342)
What would that look like. Is there an implementation of the finite difference method in biology somehow?
Re:Underrated reasons to be thankful V by kragen on Saturday November 29, 2025 @07:33AM (#46087058)
I'd be surprised.
Re:Underrated reasons to be thankful V by mountainreason on Saturday November 29, 2025 @03:37PM (#46090570)
Yeah thanks, me too. So I'm wondering what point 22 is getting at. How could that even be possible?
Re:Underrated reasons to be thankful V by kragen on Sunday November 30, 2025 @12:34PM (#46098649)
Biology can use discrete methods; we have discrete neurons (and other cells), discrete chemical species like ATP, discrete amino acids, discrete genes made of discrete nucleotides, etc.
Re:Underrated reasons to be thankful V by mountainreason on Saturday December 06, 2025 @09:02AM (#46173431)
Fur sure. But I don't get how that leads to an expectation that our biology is computing and utilizing derivatives in a computational graph.

Point 22 seems to imply that the other finds it notable that that isn't happening.

Re:Underrated reasons to be thankful V by mountainreason on Sunday December 07, 2025 @09:53AM (#46182073)
Author*