My linear algebra is terrible, but I understand what's happening in Simon's example pretty well.

I sincerely doubt you possess any kind of geometric understanding of what is going on in that code just from reading the first principles variant of cosine similarity.

>If you don’t know linear algebra, the code is inscrutable

Whether you do or don't know linear algebra, code is self-explanatory. Dot product is just the sum of element-wise products. But what the heck are dot and norm to someone that doesn't.

That’s exactly my point, it ISN'T self explanatory when it’s mathematical code like this, expressed in terms of addition and multiplication. Understanding the dot product as “sum of multiplications” is an extremely shallow understanding, and in no way does it explain why taking the dot product and dividing by magnitudes (What are those? Root of sum of squares? Why?) produces anything useful.

This code says nothing about the geometric nature of what is going on, only the arithmetic. Then we might as well read assembler. Abstractions help us reason.

The whole point of my talk here was to reassure people that you don't need to understand linear algebra in order to make use of embeddings.

You either know the math or you don't, and if you don't, you're not gonna get any new magic intuitions no matter how it's shuffled around.

>What are those? Root of sum of squares? Why?

I see what you mean. Probably can be argued that learning and applying a specific concept doesn't necessarily imply learning the deeper mathematics behind it and understanding its nature. So can just say cosine similarity measures the similarity of two vectors (lists in python) and its implementation is that sum divided by the product of sqrts of those two other sums without having to introduce and explain dot product and norm. Of course the reverse can be argued too.

I’d think you’d want to know dot and norm before embeddings. Especially since most people learn mathematical objects from the perspective of the operators that construct and manipulate them.