News for hackers. Stuff that matters.
Re:33TB of text data for a 1T-parameter model by kolinko on Saturday April 15, 2023 @09:06PM (#35586292)
Our brains are come pre-trained for speech from a few million years of evolution. Artifitial NN are trained from scratch.

A chimpanzee can listen to humans as much as it wants, and it will still not pick up much of the language.

Re:33TB of text data for a 1T-parameter model by jxy on Saturday April 15, 2023 @10:26PM (#35586688)
Right. The nature trained all the nervous systems with genetic algorithms for millennia. Individuals rather fine tuned their own with natural noise and dropoffs and natural feedback. Fine tuning our brains are rather expensive.
Re:33TB of text data for a 1T-parameter model by kevviiinn on Saturday April 15, 2023 @10:52PM (#35586823)
What makes a biological network a genetic algorithm?
Re:33TB of text data for a 1T-parameter model by chii on Saturday April 15, 2023 @11:45PM (#35587083)
The millions of years of darwinian natural selection and sexual reproduction is what makes it a genetic algorithm.
Re:33TB of text data for a 1T-parameter model by est31 on Sunday April 16, 2023 @12:27AM (#35587279)
There are a few innate reflexes but you lose them quickly and they are pretty basic. Our brains have way way more neurons/parameters than these models and are still not overfitting... the issue lies in the learning method.
Re:33TB of text data for a 1T-parameter model by og_kalu on Sunday April 16, 2023 @02:56AM (#35587880)
It's not just about reflexes though I would hardly classify reflexes as "lost quickly". That's just not true.

It's deeper than that. Our brains are optimized for a lot general human functions. Learning language is one of them. There's a whole section of the brain dedicated to it and other things.

There's also a lot of vital biological information encoded in DNA/RNA. We are not even close to starting from scratch.

Re:33TB of text data for a 1T-parameter model by est31 on Sunday April 16, 2023 @03:11AM (#35587941)
I was talking about this stuff: https://en.wikipedia.org/wiki/Primitive_reflexes
Re:33TB of text data for a 1T-parameter model by MVissers on Monday April 17, 2023 @07:56AM (#35599449)
We actually still have them as adults, but are suppressed by our brain. With brain damage or spinal cord injury, a lot of these reflexes return.