Re:Artificial Intelligence How Do Machines 'Grok' Data
by nh23423fefe on Monday April 15, 2024 @02:31PM
(#40044068)
I think its shorthand but still "accurate"
all of the following are on the same line through the origin.
8+2 1+1 2+8 3+7 4+6 5+5 6+4 7+3
and the set of lines through the origin is double-covered by the output embeddings. which means there are twice as many slices in the output embedding as in the input embedding.
so its not that 1+1 bisector points at 1 and so the sum is 1. it points at 2 because of the double cover.
this double cover is easier to see if you examine step 2.2
when a=b the leading |cos| term is 1 and the vector is roughly cos(2a),sin(2b). now go from a=b= 0 to 2pi and youll see the embedding will loop around twice as it varies from 0 to 4pi
the outside of the circle is labelled twice with 12 at top and bottom. the line through the origin picks this labelling. not the input embedding.
all of the following are on the same line through the origin.
and the set of lines through the origin is double-covered by the output embeddings. which means there are twice as many slices in the output embedding as in the input embedding.so its not that 1+1 bisector points at 1 and so the sum is 1. it points at 2 because of the double cover.
this double cover is easier to see if you examine step 2.2
and set a=b and then increment around the circle.when a=b the leading |cos| term is 1 and the vector is roughly cos(2a),sin(2b). now go from a=b= 0 to 2pi and youll see the embedding will loop around twice as it varies from 0 to 4pi
the outside of the circle is labelled twice with 12 at top and bottom. the line through the origin picks this labelling. not the input embedding.