Internal representations of LLMs encode information about truthfulness

Re:Internal representations of LLMs encode information about... by slt2021 on Wednesday October 30, 2024 @12:25PM (#41996996)

but the model doesn't operate on token directly, right? all operations are happening in the embedding space, so these tokens get mapped into manifold and one of the dimensions could be representative of fact/trivia ?

[ Parent | on HN ]

Re:Internal representations of LLMs encode information about... by ottaborra on Wednesday October 30, 2024 @12:30PM (#41997060)

tangent: any reason to assume it gets mapped to a manifold rather than something that is not?

[ Parent | on HN ]

Re:Internal representations of LLMs encode information about... by youoy on Wednesday October 30, 2024 @01:47PM (#41998107)

I think "manifolds" in AI are not the same as actual smooth manifolds. For starters I would not expect them to have locally the same dimension across the whole dataset.

[ Parent | on HN ]

Re:Internal representations of LLMs encode information about... by ottaborra on Wednesday October 30, 2024 @04:45PM (#42000066)

Something to chew on for me. But what is a manifold then if not a topological space that is locally the same as R^(some dimension) ?

[ Parent | on HN ]

Re:Internal representations of LLMs encode information about... by youoy on Wednesday October 30, 2024 @05:28PM (#42000577)

What I meant is that I can imagine cases where some part of the dataset may look like R2 and then colapse to have a spike that looks like R1, so it is not a standard manifold where all of it has the same dimension.

Appart from that, these "manifolds" have noise, so that is another difference with the standard manifolds.

[ Parent | on HN ]