News for hackers. Stuff that matters.

A comment on 33TB of text data for a 1T-parameter model. Parent comment

Re:33TB of text data for a 1T-parameter model by tgv on Sunday April 16, 2023 @04:05AM (#35588144)
> Yet we learning from 18 hours of ultra high resolution video per day (x2)

Our object recognition is trained pretty quickly. And we sleep (certainly as a child) more than 6 hours per day. But we don't learn much from just looking at pictures.

> 18 hours of reinforcement learning

You made that up.

> It makes LLMs look quite good in comparison.

So, exposing an LLM to a lot of video will make them understand language?

[ Parent | on HN ]
Re:33TB of text data for a 1T-parameter model by vkazanov on Sunday April 16, 2023 @04:38AM (#35588268)
...exposing llm to a lot of mostly labelled multimedia content
[ Parent | on HN ]
Re:33TB of text data for a 1T-parameter model by tgv on Sunday April 16, 2023 @06:26AM (#35588758)
That's not really how we learn, is it?
[ Parent | on HN ]
Re:33TB of text data for a 1T-parameter model by HPsquared on Sunday April 16, 2023 @06:49AM (#35588881)
The world as experienced by humans is unlabelled. We do, however, have instinct.
[ Parent | on HN ]
Re:33TB of text data for a 1T-parameter model by lostmsu on Monday April 17, 2023 @05:03PM (#35606575)
The poster above for some reason though encoded video is a good measure, which does not make sense actually. Brain gets raw video. With raw video it is about 100PB per year, > 200TB per day.

Even sound alone (uncompressed CD quality stereo) is 3TB/year.

[ Parent | on HN ]