News for hackers. Stuff that matters.
Re:Affine transformations by thanatropism on Wednesday January 10, 2018 @10:48AM (#16115769)
So.

In geometry an "affine vector" (c,v) is said to be the vector v tangent to c -- no longer a vector, a "tangent vector". In your neural network this would be the weights tangent to each bias. If you keep c fixed

    Tc[V] = {(c,v) | c fixed, forall v in V} 
is clearly a vector space.

Here's the fun part: if you tried to consider

    T[V] = {(c,v) | forall c in R, v in V}
this is no longer a vector space. But it's a fun structure: at each c, Tc[V] is sorta like (homeomorphic) to the Cartesian product

    {c} x V
this is a fiber bundle. For example: a torus is near each c the cartesian product of a point and a circle; a "circle squared" is a donut.

Now, we don't need to restrict ourselves to c+vx -- we can consider sigmoid(c+vx), which will have tangents (derivatives) to b at each w. For fixed c we still have a vector space with the derivatives dsigmoid/dv, and varying c you have a sigmoid fiber bundle.

There are no comments on this comment yet.