News for hackers. Stuff that matters.
Re:Chatbot Software Begins to Face Fundamental Limitations by Retric on Sunday February 02, 2025 @08:26AM (#42908491)
Hearing is also well into the terabytes worth of information per year. Add in touch, taste, smell, proprioception, etc and the brain gets a deluge.

The difference is we’re really focused on moving around in 3D space and more abstract work, where an LLM etc is optimized for a very narrow domain.

Re:Chatbot Software Begins to Face Fundamental Limitations by jdietrich on Sunday February 02, 2025 @11:40AM (#42909720)
>Hearing is also well into the terabytes worth of information per year.

If we assume that the human auditory system is equivalent to uncompressed digital recording, sure. Actual neural coding is much more efficient, so the amount of data that is meaningfully processed after multiple stages of filtering and compression is plausibly on the order of tens of gigabytes per year; the amount actually retained is plausibly in the tens of megabytes.

Don't get me wrong, the human brain is hugely impressive, but we're heavily reliant on very lossy sensory mechanisms. A few rounds of Kim's Game will powerfully reveal just how much of what we perceive is instantly discarded, even when we're paying close attention.

Re:Chatbot Software Begins to Face Fundamental Limitations by Retric on Sunday February 02, 2025 @12:51PM (#42910274)
The sensory information form individual hairs in the ear start off with a lot more data to process than simple digital encoding of two audio streams.

Neural encoding isn’t particularly efficient from a pure data standpoint just an energy standpoint. A given neuron not firing is information and those nerve bundles contain a lot of neurons.

Re:Chatbot Software Begins to Face Fundamental Limitations by PaulDavisThe1st on Sunday February 02, 2025 @12:56PM (#42910308)
"That's a coyote" I say, based on auditory evidence.

Pretty damn efficient coding.

Re:Chatbot Software Begins to Face Fundamental Limitations by hammock on Sunday February 02, 2025 @02:32PM (#42911064)
> Hearing is also well into the terabytes worth of information per year. Add in touch, taste, smell, proprioception, etc and the brain gets a deluge

Is that supposed to be a lot? Only a small fraction of that is committed to permanent storage.

A random server is today processing anywhere from tens of terabytes to hundreds of petabytes annually

Re:Chatbot Software Begins to Face Fundamental Limitations by Retric on Sunday February 02, 2025 @03:10PM (#42911345)
Low level details like that aren’t relevant to this discussion. Most human processing power is at the cellular level. The amount of processing power in a single finger literally dwarfs a modern data center, but we can’t leverage that to think only live.

So it’s not a question of ‘a lot’ it’s a question of orders of magnitude vs “the quantities used to train AI”

Library of congress has what 39 million books, tokenize every single one and you’re talking terabytes of training data for an LLM. We can toss blog posts etc to that pile but every word ever written by a person isn’t 20 orders of magnitude larger or anything.