News for hackers. Stuff that matters.

A comment on Introduction to Vector Similarity Search. Parent comment

Re:Introduction to Vector Similarity Search by ClassyJacket on Wednesday July 12, 2023 @01:48AM (#36690861)
>This is a sentence-transformers model: It maps sentences & paragraphs to a 384 dimensional dense vector space

Interesting. Can someone explain to me why 384 specifically? How did they arrive at that number?

[ Parent | on HN ]
Re:Introduction to Vector Similarity Search by computerex on Wednesday July 12, 2023 @02:19AM (#36691024)
It has to do with the architecture of the network used to create the embeddings. The embeddings is actually the output of the final layer of the model. The dimensionality is a function of the number of parameters in that layer.

Different models/architectures will produce different dimension embeddings.

[ Parent | on HN ]
Re:Introduction to Vector Similarity Search by bigbillheck on Wednesday July 12, 2023 @10:32AM (#36695110)
It's not a power of two but is still a nice round number (0x180).
[ Parent | on HN ]