News for hackers. Stuff that matters.
Re:What We Still Don’t Know About How A.I. Is Trained by fasterik on Wednesday March 29, 2023 @04:01PM (#35362589)
It's an open question the extent to which language models can learn the underlying meanings of words.

There was a recent podcast with Sean Carroll interviewing Raphaël Millière where they go into this topic and some of the research on it. Two examples I can remember are: 1) DALL-E had subject-specific, domain-specific neurons, 2) language models' representations of color terms encoded the geometry of the underlying color space, e.g. vectors in RGB space.

https://www.preposterousuniverse.com/podcast/2023/03/20/230-...

I don't think we should be too quick to assume how these models work. There's a lot that even the researchers don't know and these are empirical questions that can be studied.

There are no comments on this comment yet.