News for hackers. Stuff that matters.
Re:Language Models Are Injective and Hence Invertible by program_whiz on Thursday October 30, 2025 @10:07AM (#45760189)
Just to clarify, the total dimension of birthdays is 365 (Jan 1 through Dec 31), but a 768 dimension continuous vector means there are 768 numbers, each of which can have values from -1 to 1 (at whatever precision floating point can represent). 1 float has about 2B numbers between -1 and 1 iirc, so 2B ^ 768 is a lot more than 365.
Re:Language Models Are Injective and Hence Invertible by Arnt on Thursday October 30, 2025 @10:28AM (#45760381)
I may have misunderstood — don't they test for orthogonality? Orthogonality would seem to drop much of the information in the vectors.