Re:Language Models Are Injective and Hence Invertible
by nyeah on Thursday October 30, 2025 @11:45AM
(#45761321)
2^768 is much, much larger than 10^9. On the other hand there's the "birthday matching" thing. It's not 100% transparent to me which way this goes. Have you stepped through the orders of magnitude involved?
Re:Language Models Are Injective and Hence Invertible
by sigmoid10 on Wednesday November 05, 2025 @08:16AM
(#45822506)
Bear in mind that these are not base vectors at this stage (which would indeed give you 1/768). They are arbitrary linear combinations. There are exponentially many near orthogonal of these vectors for small epsilon. And epsilon is chosen pretty small in the paper.
Of course are vectors are not only points in one coordinate axes, but it still isn't that small compared to billions of samples.