> The sun feels hot on your skin.
No matter how many times you read that, you cannot understand what the experience is like.
> You can read a book about Yoga and read about the Tittibhasana pose
But by just reading you will not understand what it feels like. And unless you are in great shape and with greate balance you will fail for a while before you get it right. (which is only human).
I have read what shooting up with heroin feels like. From a few different sources. I certain that I will have no real idea unless I try it. (and I dont want to do that).
Waterboarding. I have read about it. I have seen it on tv. I am certain that is all abstract to having someone do it to you.
Hand eye cordination, balance, color, taste, pain, and so on, How we encode things is from all senses, state of mind, experiences up until that time.
We also forget and change what we remember.
Many songs takes me back to a certain time, a certain place, a certain feeling Taste is the same. Location.
The way we learn and the way we remember things is incredebily more complex than text.
But if you have shared excperiences, then when you write about it, other people will know. Most people felt the sun hot on their skin.
To different extents this is also true for animals. Now I dont think most mice can read, but they do learn with many different senses, and remeber some combination or permutation.
Huh, text definitely encodes multimodal experiences, it's just not as accurate and as rich encoding as the encodings of real sensations.
Perhaps, but only in the same sense that brown and green wax on paper "encodes" an oak tree.
Essentially, engineering the complete human body and mind including the nervous system. Seems highly intractable for the next couple of decades at least.
> A person who had never been outdoors could never experience that sensation by reading text.
I don't think the latter implies the former as obviously as you make it to be. Unless you believe in some sort of metaphysical description of human, you can certainly encode the feeling (as mentioned in another comment it will be reduced to electrical signals after all). The only question is how much storage you need for that encoding to get what precision. However, the latter statement, if true, is simply constrained by your input device to the brain, i.e. you cannot transfer your encoding to the hardware in this case a human brain via reading or listening. There could be higher bandwidth interfaces like neuralink that may do that to human brain and in the case of AI, an auxiliary device might not be needed and the encoding would be directly mmap'd.
A deaf person can use a tape recorder to record and play back a symphony but that does not encode the experience in any way the deaf person could share.
Applying this case to AI gives rise to the Chinese Room argument. LLMs’ propensity for hallucinations invite this comparison.
Personally, I think the Chinese room argument is invalid. In order for the person in the room to respond to any possible query by looking up the query in a book, the book would need to be infinite and therefore impossible as a physical object. Otherwise, if the book is supposed to describe an algorithm for the person to follow in order to compute a response, then that algorithm is the intelligent entity that is capable of understanding, and the person in the room is merely the computational substrate.
The only part where the book analogy falls down has to do with the technical implementation of LLMs, with their tokenization and their vast sets of weights. But that is merely an encoding for the training data. Books can be encoded similarly by using traditional compression algorithms (like LZMA).
Oh and the human book is surely infinite and unflawed right ?
>we keep bumping into the rough edges of LLMs with their hallucinations and faulty reasoning
Both things humans also do in excess
The Chinese Room is nonsensical. Can you point to any part of your brain that understands English ? I guess you are a Chinese Room then.
No not really. It's not even rare that a human confidently says and believes something and really has no idea what he/she's talking about.
>We say “sorry, I don’t know, let me get back to you.” LLMs cannot do this
Yeah they can. And they can do it much better than chance. They just don't do it as well as humans.
>And they do not even know which one they’re doing!
There's plenty of research that suggests this is the case.
Like you’re doing right now? People say “I don’t know” all the time. Especially children. That people also exaggerate, bluff, and outright lie is not proof that people don’t have this ability.
When people are put in situations where they will be shamed or suffer other social stigmas for admitting ignorance then we can expect them to be less than candid.
As for your links to research showing that LLMs do possess the ability of introspection, I have one question: why have we not seen this in consumer-facing tools? Are the LLMs afraid of social stigma?
Lol Okay
>When people are put in situations where they will be shamed or suffer other social stigmas for admitting ignorance then we can expect them to be less than candid.
Good thing I wasn't talking about that. There's a lot of evidence that human explanations are regularly post-hoc rationalizations they fully believe in. They're not lieing to anyone, they just fully believe the nonsense their brain has concocted.
Experiments on choice and preferences https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3196841/
Split Brain Experiments https://www.nature.com/articles/483260a
>As for your links to research showing that LLMs do possess the ability of introspection, I have one question: why have we not seen this in consumer-facing tools? Are the LLMs afraid of social stigma?
Maybe read any of them ? If you weren't interested in evidence to the contrary of your points then you could have just said so and I wouldn't have wasted my time. The 1st and 6th Links make it quite clear current post-training processes hurt calibration a lot.
And human language is a mechanism for referring to human experiences (both internally and between people). If you don't have the experiences, you're fundamentally limited in how useful human language can be to you.
I don't mean this in some "consciousness is beyond physics, qualia can't be explained" bullshit way. I just mean it in a very mechanistic way: language is like an API to our brains. The API allows us to work with objects in our brain, but it doesn't contain those objects itself. Just like you can't reproduce, say, the Linux kernel just by looking at the syscall API, you can't replace what our brains do by just replicating the language API.
OK, so you don't have qualia. But if know all the data needed to complete any tasks that can be related to this knowledge, does it matter?
When communicating between two entities with similar brains who have both had many thousands of hours of similar types of sensory experiences, yeah. When I read text I have a lot more than other text to relate it to in my mind; I bring to bear my experiences as a human in the world. The author is typically aware of this and effectively exploits this fact.
Of course it does. We immediately encode pictures/words/everything into vectors anyway. In practice we don't have great text datasets to describe many things in enough detail, but there isn't any reason we couldn't.
Cognition is not purely semantic. It is dynamic, embodied, socially distributed, culturally extended, and conscious.
LLMs are great semantic heuristic machines. But they don't even have access to those other components.
You are conflating the embedding layer in an LLM and an embedding model for semantic search.
My original point still stands: the space you've described cannot capture a full image of human cognition.
If I told you the text contained a detailed theory of FTL travel, could you ever construct the engine? Could you even prove it contained what I told you?
Can you imagine that given enough time, you'd recognize patterns in the text? Some sequences of glyphs usually follow other sequences, eventually you could deduce a grammar, and begin putting together strings of glyphs that seem statistically likely compared to the source.
You can do all the analysis you like and produce text that matches the structure and complexity of the source. A speaker of that language might even be convinced.
At what point do you start building the space ship? When do you realize the source text was fictional?
There's many untranslatable human languages across history. Famously, ancient Egyptian hieroglyphs. We had lots and lots of source text, but all context relating the text to the world had been lost. It wasnt until we found a translation on the Rosetta stone that we could understand the meaning of the language.
Text alone has historically proven to not be enough for humans to extract meaning from an unknown language. Machines might hypothetically change that but I'm not convinced.
Just think of how much effort it takes to establish bidirectional spoken communication between two people with no common language. You have to be taught the word for apple by being given an apple. There's really no exception to this.
It's not a theory of consciousness, it's a theory of quality. I don't think that something can be considered conscious that is constantly encoding and decoding things into and out of binary.
A few PB worth? Relax, HAL's got this. When it comes to information, it turns out that quantity has a quality all its own.
The taste of a specific dish, the exact feeling of nostalgia, or the full depth of a traumatic or ecstatic moment can be approximated in words but never fully captured. Language is symbolic and structured, while experience is often fluid, embodied, and multi-sensory. Even the most precise or poetic descriptions rely on shared context and personal interpretation, meaning that some aspects of experience inevitably remain untranslatable.
It is highly complex, but it can all be described.
As I'm typing this there is one reality that I'm understanding, the quality and completeness of the data fundamentally determines how well an AI system will work. and with just text this is hard to achieve and a multi modal experience is a must.
thank you for explaining in very simple terms where I could understand