33TB of text data for a 1T-parameter model

Re:33TB of text data for a 1T-parameter model by mindvirus on Saturday April 15, 2023 @08:34PM (#35586076)

Fair point, but accurate versus efficient are different. While humans do acquire language much more efficiently, these models "know" more than a human about most things - in that if you asked it to describe each of the 100 most popular subjects or books it would have no trouble doing so (in a dozen languages no less). So I don't think it's quite an apples to apples comparison.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by lisper on Saturday April 15, 2023 @08:41PM (#35586124)

> these models "know" more than a human about most things

So does Wikipedia. But that's not a good model of the human brain either.

> I don't think it's quite an apples to apples comparison.

Yes. That is exactly my point. Despite the superficially similar I/O behavior, the two systems are very different under the hood.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by fastball on Saturday April 15, 2023 @09:12PM (#35586324)

I don't think anyone has argued that LLMs are effectively digital humans or digital human brains, so seems like you're strawmanning a bit here.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by krainboltgreene on Sunday April 16, 2023 @01:59AM (#35587653)

You aren't reading enough of HN/Twitter then.

[ Parent | on HN ]

Re:33TB of text data for a 1T-parameter model by int_19h on Monday April 17, 2023 @06:05PM (#35607248)

Twitter is a madhouse, but as for HN, the closest I've seen anyone come to that here is claiming that the difference in underlying structure can/does lead to emergence of the same phenomena.

[ Parent | on HN ]