News for hackers. Stuff that matters.
Re:Chomsky on What ChatGPT Is Good For by krackers on Sunday May 25, 2025 @08:20PM (#44092626)
>is something implicitly present in both our minds and an LLM

The LLM doesn't start with any real structure besides the network of ops though. If there is any induced structure, it's learnable from the data. And given enough data the base network is sufficient to learn the "grammar" of not just human language but more complex CFGs and things you wouldn't traditionally consider "languages" as well (e.g. audio, images). In a sort of chicken/egg scenario, the morasses of data gives rise to the structures needed to parse and generate that data.

There are no comments on this comment yet.