News for hackers. Stuff that matters.
Re:Language Models Pack Billions of Concepts into 12,000 Dim... by aabhay on Monday September 15, 2025 @04:33AM (#45247430)
You’re right. I gave the wrong number. My model implies 2^12000 concepts, because you choose whether or not to include each concept to form your dimensional subspace.

I’m not even referring to the values within that subspace yet, and so once you pick a concept you still get the N degrees of freedom to create a complex manifold.

The main value of the mental model is to build an intuition for how “sparse” high dimensional vectors are without resorting to a 3D sphere.

There are no comments on this comment yet.