33TB of text data for a 1T-parameter model

Re:33TB of text data for a 1T-parameter model by nl on Sunday April 16, 2023 @02:31AM (#35587784)

I think that's generally accepted to be that humans have a very large language center in the brain; IE: and inductive bias towards learning language (and I think there is something about our mouth and tongue being able to make specific sounds?)

It doesn't say anything about that knowledge being precoded.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by marginalia_nu on Sunday April 16, 2023 @02:49AM (#35587857)

Isn't a language center by itself precoded human toward the acquisition of human language (which has co-evolved to make use of our language center)? I don't understand how it could be any other way.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by nl on Monday April 17, 2023 @12:35AM (#35596899)

Exactly. But it is unclear if there is intrinsic language knowledge somehow encoded too (which is what the OP is implying: https://news.ycombinator.com/item?id=35590091)

[ Parent | on HN ]