News for hackers. Stuff that matters.
Re:GPT-3 by akavi on Wednesday June 24, 2020 @01:47PM (#23631315)
> Humans achieve their performance levels while only ever observing a tiny fraction of that training data

Individual humans only ever observe a tiny fraction of that training data, but our brains are not the products of solely our individual experiences. We come out of the womb with a massive amount of evolutionarily trained, genetically encoded weights in our neural network.

Re:GPT-3 by roca on Wednesday June 24, 2020 @05:55PM (#23634309)
Where are those weights encoded? It doesn't seem like the genome has enough bits to encode a lot of weights.

I agree with other people in this thread that structure/architecture is encoded, but not weights.