Automatic Differentiation via Contour Integration

Re:Automatic Differentiation via Contour Integration by yorwba on Saturday January 18, 2020 @10:31AM (#22084933)

The discrete approximation to the integral involves sampling the function at a finite number of points and then computing a weighted sum of the sampled function values. In the case where N = 2, θ only takes the values 0 and π and the approximation is f'(x₀) ≈ 1/2 * (f(x₀+1) - f(x₀-1)), i.e. approximating the derivative with a simple difference.

So I don't think this is more biologically plausible than neurons using finite differences to do gradient descent.

[ Parent | on HN ]

Re:Automatic Differentiation via Contour Integration by mpoteat on Saturday January 18, 2020 @08:21PM (#22088682)

There's evidence that signal differences in neutrons in the visual cortex is used for object recognition, whose behavior gives rise to some of the optical illusions we experience. Weak evidence for temporal differentiation also having the same underlying mechanism.

[ Parent | on HN ]

Re:Automatic Differentiation via Contour Integration by throwawayhhakdl on Saturday January 18, 2020 @10:54AM (#22085042)

Just talking out of my ass here but it feels likely to me that the natural algorithm would be randomly learn to parse n features, and then do a sort of multi arm bandit process to determine which features to keep and also how to use them via positive and negative emotional/hormonal triggers.

Neural nets use back prop to infer bottoms up what features define what features define what features which are eventually useful for a final prediction. But we don’t really see evidence of backstop in biology.

That is to say, biology probably uses an algorithm to passively identify patterns arbitrarily, unlike neural nets which attempt to identify the patterns that solve for a given task. The latter just theoretically requires a lot of information before a space can be well defined. The former doesn’t.

[ Parent | on HN ]