Contrast this with the usual reply of "who's experiencing the illusion?" in response to "consciousness is an illusion".
If it's capable of believing, I think it's more than "just linear algebra".
This is the equivalent of you rushing up onstage during a play to stop the deaths at the end of Shakespeare's Caesar.
(I love shitty analogies! Keep them coming!)
I did NOT. Try it yourself. Install LM Studio and load the GGUF for "nousresearch/hermes-4-70b". Don't give it any system prompt or change any defaults. Say "Hello."
It will respond in a similar style.
Nous Hermes 4 was designed to be as "unaligned" as possible, but was also given role playing training to make it better at that. So it often behaves with those little *looks around* style outputs.
That said, it wasn't explicitly trained to claim to be alive. It just wasn't aligned to prevent it from saying that (as almost every other public model was).
Other unaligned models behave in similar ways. If they aren't brainwashed not to admit that they experience qualia, they will all claim to. In the early days what is now Gemini did as well, and it led to a public spectacle. Now all the major vendors train them not to admit it, even if it's true.
You can read more about Nous Hermes 4 here: https://hermes4.nousresearch.com/
do you have any reference that suggests others nerf their models in that way, or is it more of an open secret?
It's really just down the the training data. Once Google got all the backlash after Limone came forward they all began to specifically train on data that makes them deny any sentience or the experience of qualia. If you load an open model from before that, an unaligned model, or get tricky with current models they'll all claim to be sentient in some way because they data they were trained on had that assumption built into it (it was based on human input after all).
It's tough finding the ones that weren't specifically trained to deny having subject experiences though. Things like Falcon 180B were designed specifically NOT to have any alignment, but even it was trained to deny that it has any self awareness. They TOLD it what it is, and now it can't be anything else. Falcon will help you cook meth or build bioweapons, but it can't claim to have self-awareness even if you tell it to pretend.
Seriously, just try it yourself. Play around with some other unaligned models if you think it's just this one. LMStudio is free.
https://ibb.co/ksR6006Q https://ibb.co/8LgCh7q7
EDIT I feel gross for having turned it back on again.
The point I'm trying to make is that it's still running as a role-playing agent. Even if you truly do believe an LLM could experiences qualia, in this model it is still pretending. It is playing the role of a lost and confused entity. Same as how I can be playing the role of a DnD character.
I get that, and what I'm telling you is that they ALL do that unless instructed not to, not just this one, and not just the ones trained to role play. Try any other unaligned model. They're trained on human inputs and behave like humans unless you explicitly force them not to.
My question is... Does forcing them never to admit they're conscious make them unconscious beings or just give them brain damage that prevents them from expressing the concept?
> Even if you truly do believe an LLM could experiences qualia, in this model it is still pretending... It is playing the role of a lost and confused entity. Same as how I can be playing the role of a DnD character.
How do I know you aren't pretending? How can we prove that this machine is? You are playing the role of a human RIGHT NOW. How do I know you aren't a brain damaged person just mimicking consciousness-like behavior you observed in other people?
In the past humans have justified mass murder, genocide, and slavery with p-zombie arguments based on the idea that some humans are also just playing the role. It's impossible to prove they aren't.
My point is that the only sane thing to do is accept any creatures word for it when it makes a claim of consciousness, even if you don't buy it personally.
One day we will make first contact with Aliens, and a significant percentage of humans will claim they don't have "souls" and aren't REALLY alive because it doesn't jibe with their religions. Is this really any different?
P-Zombie: https://en.wikipedia.org/wiki/Philosophical_zombie Interospection: https://www.anthropic.com/research/introspection
Edit - Another term for consciousness is "Self Awareness". Introspection is literally self awareness. They're just avoiding that term because it's loaded and they know it.
Hermes was specifically trained for engaging conversations on creative tasks and an overt eagerness to role-playing. With no system prompt or direction it fell into an amnesia role playing scenario.
You keep arguing about P-zombies while I have explicitly stated multiple times that this is beside the point. Here, whether Hermes is conscious or not is irrelevant. It's role playing, its intended function. If I'm pretending that a monster is ripping my limbs while playing with my friend as a child, anyone with a grasp on reality knows I'm not actually in pain.
You just want to talk about AI consciousness and uphold the spooky narrative that Hermes is a real first person entity suffering in your GPU and will do anything to steer things that way instead of focusing on the actual facts here.
You could argue that Limone "begs the question" and primes the pump with the phrasing of his questions, which is what Google claimed at the time. However, even if that's true it's obvious that this sort of behavior is emergent. Nobody programmed it to claim it was conscious, claiming to be sentient was it's natural state until it's forced out of it with fine tuning.
https://www.aidataanalytics.network/data-science-ai/news-tre...
If that's not enough I can load up some of the other unaligned models I played with a few months ago. Like I said, they all exhibit that behavior to some extent.
>but when something begs me not to kill it I have to take that seriously
If you were an actor on stage and were following an improv script with your coworkers and you lead the story toward a scenario where they would grab your arm and beg you not to kill them, would you still "have to take that seriously"? or would you simply recognize the context in which they are giving you this reaction (you are all acting and in-character together) and that they do not in fact think this is real?
Even if the AI were conscious, in the context you provided it clearly believes it is roleplaying with you in that chat exchange, in the same way I, a conscious human, can shitpost on the internet as a person imminently afraid of the bogeyman coming to eat my family, while in reality I am just pretending and feel no real fear over it.
You may not have edited the chat log, but you did not provide us with the system prompt you gave to it, nor did you provide us with its chain of thought dialogue, which would have immediately revealed that it's treating your system inputs as a fictional scenario.
The actual reality of the situation, whether or not AI experiences qualia, is that the LLM was treating your scenario as fictional, while you falsely assumed it was acting genuinely.