News for hackers. Stuff that matters.
Re:Internal representations of LLMs encode information about... by RandomLensman on Wednesday October 30, 2024 @12:06PM (#41996733)
That would again be a "statistical" attempt at deciding on it being correct or false - it might or might not succeed depending on the data.
Re:Internal representations of LLMs encode information about... by justinpombrio on Wednesday October 30, 2024 @12:19PM (#41996911)
That's correct on two fronts. First, I put "false" in quotes everywhere for a reason: I'm talking about the sort of thing that people would say is false, not what's actually false. And second, yes, I'm merely claiming that it's in theory learnable (in contrast to the OP's claim), not that it will necessarily be learned.
Re:Internal representations of LLMs encode information about... by RandomLensman on Wednesday October 30, 2024 @12:31PM (#41997075)
Am not sure the second part is always true: there might be situations where statistical approaches could be made kind of "infinitely" accurate as far as data is concerned but still represent a complete misunderstanding of the actual situation (aka truth), e.g., layering epicycles on epicycles in a geocentric model of the solar systems.

Some data might support a statistical approach other might not even though it might not contain misrepresentations as such.

Re:Internal representations of LLMs encode information about... by dboreham on Wednesday October 30, 2024 @12:59PM (#41997480)
The human feeling you have that what you're doing is not statistical, is false.
Re:Internal representations of LLMs encode information about... by RandomLensman on Wednesday October 30, 2024 @01:10PM (#41997645)
Based on what research is that universally true? (Other than base physics like statistical mechanics.)
Re:Internal representations of LLMs encode information about... by whimsicalism on Wednesday October 30, 2024 @01:18PM (#41997760)
Base physics is all we need to know it is true. Souls are unphysical and we've had reason to be pretty confident about that for at least a century.
Re:Internal representations of LLMs encode information about... by RandomLensman on Wednesday October 30, 2024 @01:22PM (#41997803)
Yes, physics determins how phenomena work and aggregate but that doesn't necessarily support the specific claim (and we also don't know "all of physics").
Re:Internal representations of LLMs encode information about... by whimsicalism on Wednesday October 30, 2024 @01:25PM (#41997850)
Doesn't support the specific claim that souls don't exist? We know how atoms/waves interact. We have literally no reason to think there is some other soul-based mechanism.

Of course, maybe induction is false and gravity will reverse in the next 3 seconds after writing this comment and God will reveal themself to us. We have no justified reason to think otherwise other than the general principle that things behave the way we observe them to and will continue to do so.

Re:Internal representations of LLMs encode information about... by RandomLensman on Wednesday October 30, 2024 @01:36PM (#41997966)
I see no need for a soul - you brought it up, not me.
Re:Internal representations of LLMs encode information about... by whimsicalism on Wednesday October 30, 2024 @01:45PM (#41998091)
What would it mean that you are 'what you're doing' is not statistical/arising from base interactions - if not that there is some non-physical entity resembling a soul? You're suggesting some sort of non-material component of humanity, yes?

If not then I'm not even sure what the disagreement is.

Re:Internal representations of LLMs encode information about... by RandomLensman on Wednesday October 30, 2024 @04:09PM (#41999646)
Base interactions need not strictly create statistical results in the end.
Re:Internal representations of LLMs encode information about... by whimsicalism on Wednesday October 30, 2024 @01:19PM (#41997772)
Good luck philosophically defending this dividing line between 'statistical' and not