Re:Jina AI Launches First Open-Source 8K Text Embedding, Riv...
by karxxm on Thursday October 26, 2023 @08:42AM
(#38024758)
Unfortunately this is not feasible with a large amount of words due to the quadratic scaling. But thanks for the response!
[ Parent
| on HN ]
Re:Jina AI Launches First Open-Source 8K Text Embedding, Riv...
by minimaxir on Thursday October 26, 2023 @12:20PM
(#38027799)
Not sure what you mean by large amount of words. You can fit a PCA on millions of vectors relatively performantly, then inference from it is just a matmul.
Re:Jina AI Launches First Open-Source 8K Text Embedding, Riv...
by karxxm on Saturday October 28, 2023 @04:20PM
(#38053038)
Not true. You need a distance matrix (for classical PCA it's a covariance matrix), which scales quadratically with the number of points you want to compare.
If you have 1 Mio. vectors, each creating a float entry in the matrix, you will end up with approx (10^6)^2 / 2 unique values, which is roughly 2000Gb of memory.