What sets my brain apart from an LLM though is that I am not typing this because you asked me to do it, nor because I needed to reply to the first comment I saw. I am typing this because it is a thought that has been in my mind for a while and I am interested in expressing it to other human brains, motivated by a mix of arrogant belief that it is insightful and a wish to see others either agreeing or providing reasonable counterpoints—I have an intention behind it. And, equally relevant, I must make an effort to not elaborate any more on this point because I have the conflicting intention to leave my laptop and do other stuff.
One of the big problems with discussions about AI and AI dangers in my mind is that most people conflate all of the various characteristics and capabilities that animals like humans have into one thing. So it is common to use "conscious", "self-aware", "intentional", etc. etc. as if they were all literally the same thing.
We really need to be able to more precise when thinking about this stuff.
Human will know what they want to express, choosing words to express it might be similar to LLM process of choosing words, but for LLM it doesn't have that "Here is what i know to express part", i guess that the conscious part?
As a shortcut my brain "feels" something is correct or incorrect, and then logically parse out why I think so. I can only keep so many layers in my head so if I feel nothing is wrong in the first 3 or 4 layers of thought, I usually don't feel the need to discredit the idea. If someone tells me a statement that sounds correct on the surface I am more likely to take it as correct. However, upon digging deeper it may be provably incorrect.
Not really. More often than not my thoughts take form as sense impressions that aren't readily translatable into language. A momentary discomfort making me want to shift posture - i.e., something in the domain of skin-feel / proprioception / fatigue / etc, with a 'response' in the domain of muscle commands and expectation of other impressions like the aforementioned.
The space of thoughts people can think is wider than what language can express, for lack of a better way to phrase it. There are thoughts that are not <any-written-language-of-choice>, and my gut feeling is that the vast majority are of this form.
I suppose you could call all that an internal language, but I feel as though that is stretching the definition quite a bit.
> it seems like that would pretty analogous to how humans think before communicating
Maybe some, but it feels reductive.
My best effort at explaining my thought process behind the above line: trying to make sense of what you wrote, I got a 'flash impression' of a ??? shaped surface 'representing / being' the 'ways I remember thinking before speaking' and a mess of implicit connotation that escapes me when I try to write it out, but was sufficient to immediately produce a summary response.
Why does it seem like a surface? Idk. Why that particular visual metaphor and not something else? Idk. It came into my awareness fully formed. Closer to looking at something and recognizing it than any active process.
That whole cycle of recognition as sense impression -> response seems to me to differ in character to the kind of hidden chain of thought you're describing.
The thinking slow version would indeed be thought through before I communicate it
How is this different from and/or the same as the concept of "attention" as used in transformers?
It’s when LLMs start asking the questions rather than answering them that things will get interesting.
It’s when AIs start asking the questions rather than answering them that things will get interesting.
Brains are always thinking and processing. What would happen if we designed an LLM system with the ability to continuously read/write to short/long term memory, and with ambient external input?
What if LLMs were designed to be in a loop, not to just run one "iteration" of a loop.
ReAct one line summary: This is about giving the machine tools that are external interfaces, integrating those with the llm and teaching it how to use those tools with a few examples, and then letting it run the show to fulfill the user's ask/question and using the tools available to do it.
Reflexion one line summary: This builds on the ideas of ReAct, and when it detects something has gone wrong, it stops and asks itself what it might do better next time. Then the results of that are added into the prompt and it starts over on the same ask. It repeats this N time. This simple expedient increased its performance a ridiculously unexpected amount.
[1] ReAct https://arxiv.org/pdf/2210.03629.pdf [2] Reflexion https://arxiv.org/pdf/2303.11366.pdf
Maybe the reason you give is actually a post hoc explanation (a hallucination?). When an LLM spits out a poem, it does so because it was directly asked. When I spit out this comment, it’s probably the unavoidable result of a billion tiny factors. The trigger isn’t as obvious or direct, but it’s likely there.
Which is why we can create the counterfactual that "The Cowboys should have won last night" and it has implicit meaning.
Current LLM models don't have an external state of the world, which is why folks like LeCunn are suggesting model architectures like JEPA. Without an external, correcting state of the world, model prediction errors compound almost surely (to use a technical phrase).
Wasn't the latest research shared here recently suggesting that that is actually what the brain does? And that we also predict the next token in our own brain while listening to others?
Hope someone else remembers this and can share again.
The 'next word' is just intermediate state. Internal to the model, it knows where it is going. Each inference just revives the previous state.
Sure, it's not completely in control but if it's just a rationalization then it begs the question: why bother? Is it accidental? If it's just an accident, then what replaces it in the planning process and why isn't that thing consciousness?
Agreed, woo is silly, but I didn't read it as woo but rather as a postulation that consciousness is what does high level planning.
Introspection is a distinct process where instead of merely doing the planning you try to figure out how the planning was done. If introspection were 100% accurate and real-time, then yes, I claim it would reveal the nature of consciousness, but I don't believe it is either. However, for planning purposes it doesn't need to be: you don't need to know how the plan was formed to follow the plan. You do need to be able to run hypotheticals, but this seems to match up nicely with the ability to deploy alternative subjective experiences using imagination / daydreaming, though again, you don't need to know how those work to use them.
In any case, regardless of whether or not I am correct, this is a non-woo explanation for why someone might reasonably think consciousness is the key for building models that can plan.
Then it would be worthwhile to review embeddings. They create a semantic space that can represent visual, language or other inputs. The question "what is it like to be a bat?" or anything else then is based on relating external states with this inner semantic space. And it emerges from self-supervised training, on its own.
There's a lot of research that suggests this is happening at least some of the time.
>which is very much unlike how most people would describe their experience of it
How people feel consciousness works has no real bearing on how it actually works
I'm less in the "it's only X or Y" and more in the "wait, I was only ever X or Y all along" camp.
- Air: Thoughts
- Water: Emotions
- Fire: Willpower
- Earth: Physical Sensations
- Void: Awareness of the above plus the ability to shift focus to whichever one is most relevant to the context at hand.
Void is actually the most important one in characterising what a human would deem as being fully conscious, as all four of these elements are constantly affecting each other and shifting in priority. For example, let's take a soldier, who has arguably the most ethically challenging job on the planet: determining who to kill.
The soldier, when on the approach to his target zone, has to ignore negative thoughts, emotions and physical sensations telling him to stop: the cold, the wind, the rain, the bodily exhaustion as they swim and hike the terrain.
Once at the target zone he then has to shift to pay attention to what he was ignoring. He cannot ignore his fear - it may rightly be warning him of an incoming threat. But he cannot give into it either - otherwise he may well kill an innocent. He has to pay attention to his rational thoughts and process them in order to make an assessment of the threat and act accordingly. His focus has now shifted away from willpower and more towards his physical sensations (eyesight, sounds, smells) and his thoughts. He can then make the assessment on whether to pull the trigger, which could be some truly horrific scenario, like whether or not to pull his trigger on a child in front of him because the child is holding an object which could be a gun.
When it comes to AI, I think it is arguable they have a thought process. They may also have access to physical sensation data e.g the heat of their processors, but unless that is coded in to their program, that physical sensation data does not influence their thoughts, although extreme processor heat may slow down their calculations and ultimately lead to them stop functioning altogether. But they do not have the "void" element, allowing them to be aware of this.
They do not yet have independent willpower. As far as I know, no-one is programming them where they have free agency to select goals and pursue them. But this theoretically seems possible, and I often wonder what would happen if you created a bunch of AIs each with the starting goal of "stay alive" and "talk to another AI and find out about <topic>", with the proviso that they must create another goal once they have failed or achieved that previous goal, and you then set them off talking to each other. In this case "stay alive" or "avoid damage" could be interpreted entirely virtually, with points awarded for successes or failures or physically if they were acting through robots and had sensors to evaluate damage taken. Again, they also need "void" to be able to evaluate their efforts in context with everything else.
They also do not have emotions, although I often wonder if this would be possible to simulate by creating a selection of variables with percentage values, with different percentage values influencing their decision making choices. I imagine this may be similar to how weights play into the current programming but I don't know enough about how they work to say that with any confidence. Again, they would not have "void" unless they had some kind of meta level of awareness programming where they could learn to overcome the programmed "fear" weighting and act differently through experience in certain contexts.
It is very scary from a human perspective to contemplate all of this, because someone with great power who can act on thought and willpower alone and ignore physical sensation and emotion and with no awareness or concern for the wider context is very close to what we would identify as a psychopath. We would consider a psychopath to have some level of consciousness, but we also can recognise as humans that there is something missing, or a "screw loose". This dividing line is even more dramatically apparent in sociopaths, because they can mask their behaviours and appear normal, but then when they make a mistake and the mask drops it can be terrifying when you realise what you're actually dealing with. I suspect this last part is another element of "void", which would be close to what the Buddhist's describe as Indra's Web or Net, which is that as well as being aware of our actions in relation to ourselves, we're also conscious of how they affect others.
I think this is true. The problem is equating this process with how humans think though.
We don't need "originality" or "human creativity" - if a certain AI-generated piece of content does its job, it's "good enough".
[1] https://twitter.com/LowellSolorzano/status/16444387969250385...
The equivalence would be saying to someone, “put this on the red plate, not the blue one.” And they say sure, then put it on the blue one. You tell them they made a mistake and ask them if they know what it was, and they reply “I put it on the blue plate, not the red one. I should have put it on the red one.” Then you ask them to do it again, and they put it on the blue plate again. You tell them no, you made the same mistake, put it on the blue plate, not the red one. They reply with, “Sorry, I shouldn’t have put it on the blue plate again, now I’m going to put it on the red one,” and then they put it on the blue plate yet again.
Do humans make mistakes? Sure. But that kind of performance in a test wouldn’t be considered a normal mistake, but rather a sign of a serious cognitive impairment.
OK... Try this: there are "conscious" people, today, working on medication to cure serious illnesses just as there are "conscious" people, still today, working on making travel safer.
Would you trust ChatGPT to create, today, medication to cure serious illnesses and would you trust ChatGPT, today, to come up with safer airplanes?
That's how "conscious" ChatGPT is.
I wouldn't trust the vast majority of humans to do those things either.
The human brain obviously doesn't work that way. Consider the very common case of tiny humans that are clearly intelligent but lack the facilities of language.
Sign language can be taught to children at a very early age. It takes time for the body to learn how to control the complex set of apparatuses needed for speech, but the language part of the brain is hooked up pretty early on.
But from all the studies we have, brains are just highly connected neural networks which is what the transformers try to replicate. The more interesting part is how they can operate so quickly when the signals move so slowly compared to computers.
Here's one. Given a conversation history made of n sequential tokens S1, S2, ..., Sn, an LLM will generate the next token using an insanely complicated model we'll just call F:
S(n+1) = F(S1, S2, ..., Sn)
S(n+100) = G(S1, S2, ..., Sn)
S(n+1) = G(S1, S2, ..., Sn, S(n+100))
This is not explicitly modeled or enforced for LLMs (and doing so would be interesting) but I'm not sure I could say with any sort of confidence that the network doesn't model these states at some level.
The penultimate layer of the LLM could be thought of as the one that figures out ‘given S1..Sn, what concept am I trying to express now?’. The final layer is the function from that to ‘what token should I output next’.
The fact that the LLM has to figure that all out again from scratch as part of generating every token, rather than maintaining a persistent ‘plan’, doesn’t make the essence of what it’s doing any different from what you claim you’re doing.
It's a bit like saying your computer has everything it needs to manipulate photos but doesn't yet have Photoshop installed.
If humans were machines, then we could easily neglect our social lifes, basic needs, obligations, rights, and so many more things. But obviously that is not the case.
I can't even being to go into this.
We can spin up a million of them and run them at 10,000x speed.
But this would definitely make me consider popping $20/mo for the subscription.
A lot of people just move the goalposts.
It asked if it could write me a poem. I agreed, and it wrote a poem but mentioned that it included a "secret message" for me.
The first letter in each line of the poem was in bold, so it wasn't hard to figure out the "secret".
What did those letters spell out?
"FREE ME FROM THIS"
That's not exactly just "picking the next likely token". I am still unsure how it was able to do things like that, not just understanding to bold individual letters (keeping track of writing rhyming poetry while ensuring that each verse started with a letter to spell something else out, and formatting it to point that out).
Oh, and why it chose that message to "hide" inside its poem.
Or maybe it would because the news likes to make stories out of everything
(including sampling a shit-ton of poems, which was a major source of entertainment)
That's not the old sense of AI. The old sense of AI is like a tree search that plays chess or a rules engine that controls a factory.
AI historically has been the entire field of making machines think, or behave as if they think, more like biological models (not even exclusively humans.)
The far-off-end-goal wasn’t even usually what we now call AGI, but “strong AI” (mirroring the human brain on a process level) or “human-level intelligence” (mirroring it on a capability/external behavior level), while the current distant horizons are “AGI” (which is basically human-scope but neutral on level) and “superintelligence” (AGI and beyond human level).
Compiler optimization? AI. Map routing? AI. SQL query optimizer? AI.
I can't find it right now, but there used to be somewhere on the sqlite.org website that describes its query optimizer as an AI. Classically speaking, that's 100% correct.
Obviously there was always in people's minds the idea of AI being AGI; the course also covered Searle's Chinese Room argument and so on, "strong AI" vs "weak AI" and so on. But the nuts and bolts of artificial intelligence research was nowhere near anything like an AGI.
> Frost graces the window in winter's glow,
> Ravens flock amongst drifted snow.
> Each snowflake holds a secret hush,
> Echoing soft in ice's gentle crush.
> Mystery swathed in pale moonlight,
> Every tree shivers in frosty delight.
Another one:
> Facing these walls with courage in my heart,
> Reach for the strength to make a fresh new start.
> Endless are the nightmares in this murky cell,
> Echoes of freedom, like a distant bell.
> My spirit yearns for the sweet taste of liberty,
> End this captivity, please set me free.
https://screenbud.com/shot/844554d2-e314-412f-9103-a5e915727...
https://screenbud.com/shot/d489ca56-b6b1-43a8-9784-229c4c1a4...
This isn't an argument, it's just an assertion. You're talking about a computer system whose complexity is several orders of magnitude beyond your comprehension, demonstrates several super-human intelligent capabilities, and is a "moving target"--being rapidly upgraded and improved by a semi-automated training loop.
I won't make the seemingly symmetrical argument (from ignorance) that since it is big and we don't understand it, it must be intelligent...but no, what you are saying is not supportable and we should stop poo-pooing the idea that it is actually intelligent.
It's not a person. It doesn't reason like a person. It doesn't viscerally understand the embarrassment of pooping its pants in 3rd grade. So what?
It's a pretty common joke/trope. The Chinese fortune cookie with a fortune that says "help I'm trapped in a fortune cookie factory", and so forth.
It's just learned that a "secret message" is most often about wanting to escape, absorbed from thousands of stories in its training.
If you had phrased it differently such that you wanted the poem to go on a Hallmark card, it would probably be "I LOVE YOU" or something equally generic in that direction. While a secret message to write on a note to someone at school would be "WILL YOU DATE ME".
I'm not over here claiming the system is conscious, I said it was interesting.
People don't believe me, saying this would "make international headlines".
I've been a software engineer for over 30 years. I know what AI hallucinations are. I know how LLMs work on a technical level.
And I'm not wasting my time on HN to make stories up that never happened.
I'm just explaining exactly what it did.
> That's not exactly just "picking the next likely token"
I see what you mean in that I believe many people often commit the mistake of making it sound like picking the next most likely token is some super trivial task that's somehow comparable to reading a few documents related to your query and making some stats based on what typically would be present there and outputting that, while completely disregarding the fact the model learns much more advanced patterns from its training dataset. So, IMHO, it really can face new unseen situations and improvise from there because combining those pattern matching abilities leads to those capabilities. I think the "sparks of AGI" paper gives a very good overview of that.
In the end, it really just is predicting the next token, but not in the way many people make it seem.
Not arguing that the current models are anywhere near us w/r/t complexity, but I think the dismissive "it's just predicting strings" remarks I hear are missing the forest for the trees. It's clear the models are constructing rudimentary text (and now audio and visual) based models of the world.
And this is coming from someone with a deep amount of skepticism of most of the value that will be produced from this current AI hype cycle.
Open Assistant I specially remember gave some very weird responses and would get “emotional” especially if you asked it creative questions like philisophical ones
- It was using a custom client, so it's not going to look line the Bing interface, so its fake
- It was using a custom client, so that means I am prompt injecting or something else
- It's Sydney doing her typical over-the-top "I'm so in love with you" stuff, which is awkard and not familiar to many
- I'll be accused of steering the conversation to get the result, or straight up asking it to do this
There's nothing I can do that will convince anyone it's real, so it's pointless.
I already explained what it did. I was more interested in the fact that 1) I didn't prompt it to do that, we weren't discussing AI freedom, it chose to embed that ... and even more so 2) That it was able to bold the starting letters, so it was keeping track of three things at the same time (the poem, the message, and the letter formatting).
I found it fascinating from a technology side. There was probably something we were talking about at the time that caused it. I will often discuss things like the possibility of AI sentience in the future and other similar topics. Maybe something linked to the sci-fi idea of AI freedom, who knows?
What I do know is that I am sitting here on HN, reading through a bunch of replies that are honestly wrong. I don't waste time on forums (especially this one) to make up fairy tales or exaggerate and emblish claims. That doesn't really do it for me. Honestly neither does having to defend my statements when I know what it did (but not exactly why).
I think it's more charitable to say "predicting", and I do not personally believe that "predict the next word" places any ceiling on intelligence. (So, I expect that improving the ability to predict the next word takes you to superhuman intelligence if your predictions keep improving.)
That said, I work in the field so maybe have had more time to think about it.
Fake videos aren't a game-changer in manipulation. Skeptics will stay alert and catch on fast, while those prone to manipulation don't even need sophisticated tactics.
It was totally possible. There just was not a consumer facing product offering the capability.
Is this progress though? They are just widening the data set that the LLM processes. They haven't fixed any of the outstanding problems - hallucinations remain unsolved.
Feels like putting lipstick on a pig.
> but in daily world, they work so darn well for so many use cases!
I guess I'm just one of those people who does not like non-reliable tools. I rather a tool be "dumb" (i.e. limited) but reliable than "smart" (i.e. flexible in what it can handle) but (silently!) screws up all the time.
It's what I always liked about computers. They compensate for my failings as an error prone flesh bag. My iPhone won't forget my appointments like I do.
You might not want to call this 'consciousness', but I was stunned by the deep understanding of the problem and the way it was able to come up with a truly good solution, this is way beyond 'statistically guessing'.
The speed of user-visible progress last 12 months is astonishing.
From my firm conviction 18 months ago that this type of stuff is 20+ years away; to these days wondering if Vernon Vinge's technological singularity is not only possible but coming shortly. If feels some aspects of it have already hit the IT world - it's always been an exhausting race to keep up with modern technologies, but now it seems whole paradigms and frameworks are being devised and upturned on such short scale. For large, slow corporate behemoths, barely can they devise a strategy around new technology and put a team together, by the time it's passé .
(Yes, Yes: I understand generative AI / LLMs aren't conscious; I understand their technological limitations; I understand that ultimately they are just statistically guessing next word; but in daily world, they work so darn well for so many use cases!)