It can measure them and process it as a number, but when does sensing come into play? Does a red sheet of paper recorded by a simple camera evoke the same "feeling" of redness inside the camera that it evokes in a human being?
The flippant answer is "of course not, the camera has no consciousness/perception". The question is then how and when this feeling of "redness" gets created and what are the necessary conditions for it to happen.
That would come from learned data, experience, imo. A child who's never touched a red hot electric stove, for example, would not have any bias towards it. No fear, no love, unless they previously interacted with one. They would try and get close, watch it, smell it, touch it, to learn more.
The most interesting part of consciousness is how does one decide based on incomplete information? If you need to learn more, where do you find the raw information? It seems to be done subconsciously, some people have a higher affinity for learning/fact-finding than others. But all people are pre-wired to learn from others, distributed computing works best heh.
I guess you're asking the same question as me, where is this "programming" and how is it created? It's not just raw experience, it seems to be genetic/evolutionary. A sort of basic firmware to bootstrap further learning.
I find it fascinating, it would seem the brain never stops processing the data it acquires. During the day, and during the night, it always runs learning jobs.
When you're looking at a red sheet of paper, reading the input in the form of a signal is not the only thing happening. The signal is definitely read and passed along to other brain circuitry for processing, but somehow another thing happens: the experience of red (which is what the other poster called "feeling"). This experience of red is fundamentally different from the experience of green or blue or from the complete absence of looking. This experience is what I was referring to.
This experience seems to happen without any prior knowledge or exposure to the color red. The first time you stumble upon red light, this experience of red arises. You can tell red from green by the marked difference in experiences, but you cannot explain this difference in words to someone who hasn't experienced red or green themselves.
How does this experience get created? Does any sensor (such as a camera) have such experiences? Why not?
It's the decision making that's complicated.
To make a decision on whether it would like chocolate ice cream or not (before trying it) a brain will take all the data gathered over time, cross reference it with related data (does chocolate taste good alone? \ did I like it? \ It looked rather inedible. \ was it high cocoa content or not? \ Chocolate yoghurt seems similar, but I did not like it. \ I do like cold, but only when it's hot outside. \ My friend says chocolate ice cream tastes good. \ My other friend says it tastes bad. \ trust or discard opinions? and much more).
All within minutes, seconds even.
An AGI would need an absolutely massive database of knowledge similar to the one acquired by a human over the first 20-30 years of their life, if it's to be truly general.
It would also need bias correction, and maybe 2-5 other AGIs to form a consensus before making any decision properly.
And even that would not be enough to fit in with humans - if you want an AGI to make the objectively right decisions, it will have to ignore many human feelings/biases.
If you want it to fit in with people, it will have to make "mistakes", i.e. ineffective decisions.