Call 10^10 \approx 2^40 for convenience, and 8000 \approx 2^13, which gives us a 2^53 entropy estimate, or about a petabyte of information as an estimate of what the human brain can store (discounting more exotic theories of memory stored in DNA or some such).
[0] https://en.wikipedia.org/wiki/Human_brain#Microanatomy
[1] https://psychology.stackexchange.com/questions/7967/how-many...
A single pyramid neuron in the neocortex might be more comparable to a multilayer neural net.
https://www.biorxiv.org/content/10.1101/2021.10.25.465651v1....
We don't understand how they work at the subatomic level simply because human understanding of the subatomic world is not complete, but even just at the atomic level a single neuron is massively more complex than anything humans have created.
Going up to the molecular level, even that is staggeringly more complex than the incredibly simple abstractions that make up a neural net.
Is what happens in the brain at the molecular, atomic, or subatomic levels relevant or necessary to intelligence and consciousness? We just don't know yet, but we do know all of that is far more complex and very different from the simple abstractions that are used for neural nets and LLMs.
The back of a napkin calculations in this thread don't even begin to do justice to the tremendous amount of "calculation" or "storage" that happens in the human brain.
In contrast the data we collect through our nervous system is rich and meaningful and far deeper than just raw text. We can even manipulate the environment as we learn to facilitate faster learning eg. pick up a ball and throw it, rather than just watch videos of balls being thrown.
Our object recognition is trained pretty quickly. And we sleep (certainly as a child) more than 6 hours per day. But we don't learn much from just looking at pictures.
> 18 hours of reinforcement learning
You made that up.
> It makes LLMs look quite good in comparison.
So, exposing an LLM to a lot of video will make them understand language?
Even sound alone (uncompressed CD quality stereo) is 3TB/year.
Some deaf/blind people can read braille — they learn fine too.
Much less data than you might think.
Bare in mind most people can run the brain on 2500kCal/day.
And so on.
Which is, in turn, 2900 W·h, or 2.5 times less than one A100 card working round the clock (300W·24h)
The significant thing is the reinforcement. Without it you have to resort to the openai brainlet style 10tb of text training.
If their model had reinforcement built in then you could just plop it in front of a person or on reddit and it would rapidly self-learn on a fraction of the data. Their training model relies on it learning purely based on observations rather than interaction which is inefficient.
This is assuming we are learning nothing during sleep, which probably isn't true.
By the time a person is 21 years old, they have been trained on at least 1 petabyte of data.
By two years old, about 125TB of data. It makes LLMs look quite good in comparison.