News for hackers. Stuff that matters.
Re:Artificial Intelligence How Do Machines 'Grok' Data by nh23423fefe on Monday April 15, 2024 @02:31PM (#40044068)
I think its shorthand but still "accurate"

all of the following are on the same line through the origin.

    8+2 1+1 2+8 3+7 4+6 5+5 6+4 7+3
and the set of lines through the origin is double-covered by the output embeddings. which means there are twice as many slices in the output embedding as in the input embedding.

so its not that 1+1 bisector points at 1 and so the sum is 1. it points at 2 because of the double cover.

this double cover is easier to see if you examine step 2.2

    | cos(wk(a − b)/2)|(cos(wk(a + b)),sin(wk(a + b)))
and set a=b and then increment around the circle.

when a=b the leading |cos| term is 1 and the vector is roughly cos(2a),sin(2b). now go from a=b= 0 to 2pi and youll see the embedding will loop around twice as it varies from 0 to 4pi

the outside of the circle is labelled twice with 12 at top and bottom. the line through the origin picks this labelling. not the input embedding.

                     , - ~ 12~ - 1
                 , '               ' 2
               ,                       3
              ,                         4
             ,                           5
             ,                           6
             ,                           ,
              ,                         7
               ,                       8
                 ,                   9
                   ' - , _ 12_ 11 10

There are no comments on this comment yet.