Re:Oops! We Automated Bullshit
by TeMPOraL on Friday November 17, 2023 @01:13PM
(#38307545)
> How do I know I _know_ something? Usually because I have searched for evidence from multiple angles / dimensions / senses. When multiple independent observations of quality agree, and my conclusion matches the result, it is safe to say that I know that fact.
This is technically true, but no one actually does much of it consciously. But this is how you picked up every skill and understanding, including language, since you've been born.
And this - the unconscious fusion of correlated patterns - is pretty much what LLMs undergo in training.
EDIT:
Also, a lot of confusion is created by using terms like "a fact" or "a conclusion" or "a result", as if they were fixed points in space, fully defined mathematical symbols. But there exist no such thing. There is no binary "know/don't know", there is no binary "result match/does't match the conclusion". Those are all continuous quantities, that get rounded off for convenience.
This is technically true, but no one actually does much of it consciously. But this is how you picked up every skill and understanding, including language, since you've been born.
And this - the unconscious fusion of correlated patterns - is pretty much what LLMs undergo in training.
EDIT:
Also, a lot of confusion is created by using terms like "a fact" or "a conclusion" or "a result", as if they were fixed points in space, fully defined mathematical symbols. But there exist no such thing. There is no binary "know/don't know", there is no binary "result match/does't match the conclusion". Those are all continuous quantities, that get rounded off for convenience.