Re:Advances in semiconductors are feeding the AI boom
by zer00eyz on Thursday March 28, 2024 @12:41PM
(#39854033)
>> 100 trillion synapses (very roughly analogous to transistors?)
Not even remotely comparable
* its unlikely synapses are binary. Candidly they probably serve more than one purpose.
* transistor count is a bad proxy for other reasons. A pipeline to do floats are not going to be useful for fetch from memory. "Where" the density lies is important.
* Power: On this front transistors are a joke.
* The brain is clockless, and analog... frequency is an interesting metric
Binary systems are going to be bad at simulating complex processes. LLM's are a simulation of intelligence, like forecasting is a simulation of weather. Lorenz shows us why simulation of weather has limits, there isnt some magical math that will change those rules for ML to make the leap to "AGI"
Re:Advances in semiconductors are feeding the AI boom
by luyu_wu on Thursday March 28, 2024 @12:47PM
(#39854129)
Transistor power is really not a joke. Synapses would take far far FAR more power at close to similar frequencies. Biological neurons are incredibly inefficient.
Re:Advances in semiconductors are feeding the AI boom
by goatlover on Thursday March 28, 2024 @06:24PM
(#39858121)
Biology is incredibly efficient at what it does well though. Thus only 20 watts of brain energy to coordinate everything we do. We didn't evolve to be mentats.
Not even remotely comparable
* its unlikely synapses are binary. Candidly they probably serve more than one purpose.
* transistor count is a bad proxy for other reasons. A pipeline to do floats are not going to be useful for fetch from memory. "Where" the density lies is important.
* Power: On this front transistors are a joke.
* The brain is clockless, and analog... frequency is an interesting metric
Binary systems are going to be bad at simulating complex processes. LLM's are a simulation of intelligence, like forecasting is a simulation of weather. Lorenz shows us why simulation of weather has limits, there isnt some magical math that will change those rules for ML to make the leap to "AGI"