Re:33TB of text data for a 1T-parameter model
by MagicMoonlight on Sunday April 16, 2023 @09:19AM
(#35589814)
Most of that data is the same all day e.g vision. There’s very little difference between frames.
The significant thing is the reinforcement. Without it you have to resort to the openai brainlet style 10tb of text training.
If their model had reinforcement built in then you could just plop it in front of a person or on reddit and it would rapidly self-learn on a fraction of the data. Their training model relies on it learning purely based on observations rather than interaction which is inefficient.
The significant thing is the reinforcement. Without it you have to resort to the openai brainlet style 10tb of text training.
If their model had reinforcement built in then you could just plop it in front of a person or on reddit and it would rapidly self-learn on a fraction of the data. Their training model relies on it learning purely based on observations rather than interaction which is inefficient.