News for hackers. Stuff that matters.
Re:What Are Embeddings? by cubefox on Sunday June 25, 2023 @05:08PM (#36472607)
An array of floats (an n-dimensional vector) which represents some piece of data like a text or an image. Different embeddings can be more or less close to each other, and this closeness indicates similarity.
Re:What Are Embeddings? by teruakohatu on Sunday June 25, 2023 @09:32PM (#36474663)
^^^ this is a really succinct explanation. I would just add that an embedding is an array/vector of numbers. Does not have to be floats although they are in practice.

The magic is creating the embedding, embeddings themselves are nothing special.

Re:What Are Embeddings? by specproc on Monday June 26, 2023 @05:55AM (#36477696)
Yeah, I'd been working with them for ages before I really wrapped my head around the sausage machine. There was a Wolfram blog a while back that really helped me grok it [0]. The last state of a neural network prior to prediction, if I understand correctly.

[0] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

Re:What Are Embeddings? by chenghuzi on Monday June 26, 2023 @12:43AM (#36475802)
Not just similarity. It also includes all other possible relationships if you want, such as order, connectivity, etc.
Re:What Are Embeddings? by pedrosorio on Monday June 26, 2023 @01:33AM (#36476055)
What is order/connectivity in R^n?
Re:What Are Embeddings? by chenghuzi on Wednesday June 28, 2023 @11:12PM (#36515891)
It's pretty arbitrary. For example, you can define order as the product of all order relationships along each single dimension (binary product). As for connectivity, they can also be defined as boolean functions under certain radius, like the persistent homology spectrum.
Re:What Are Embeddings? by TeMPOraL on Monday June 26, 2023 @04:52AM (#36477332)
I'd say similarity, but any and all kinds of similarity you could ever think of - as long as you have enough dimensions and training data to feed it.

My current belief is that a hundred thousand+ dimensional latent space, as in SOTA LLMs, seems to have enough dimensions to reduce a large array of cognitive skills into a proximity search.

Re:What Are Embeddings? by LudwigNagasena on Monday June 26, 2023 @03:45AM (#36476864)
Generally, it is any transformation that preserves relevant structure.
Re:What Are Embeddings? by nborwankar on Monday June 26, 2023 @04:31PM (#36485426)
Importantly it’s a vector space with a metric. Not just an array of vectors. The original Word2Vec paper had the jaw dropping “king - man + woman -> queen” which needs the ability to do addition/subtraction on vectors. Also words “close” in meaning result in vectors “close” in some sense e.g. it needs a formal measure of distance (eg cosine similarity) which you hint at.