Chatting with an agent using language feels like talking to an assistant.
I don’t think my meat brain is able to really differentiate between writing a message to an agent versus writing an agent to a human.
I’ll keep being polite and grateful to agents, so that I maintain those social habits for when I talk to humans.
My take is that people deserve to be treated with respect because they are people, not because they are smart and/or interesting. I don't treat people differently based on their IQ.
But, as I said, I totally understand your point of view, particularly the idea that people are different from machines and deserve to be treated differently.
An unlikely failure mechanism perhaps but it costs me nothing to avoid it.
I do that. But I also act rude and call it names at times. I think that is also what makes us human.
I don’t think LLMs are conscious but I’m still nice to them because it makes me feel bad when I’m rude to them.
An LLM is not a person. Do not humanise it. Do not personalise it. Do not treat it as more than a glorified autocorrect—that’s what it really is.
Humanising LLMs is exactly what these companies want, because then they can really take advantage of the I in AI which is exactly what makes people think these steroidal spellcheckers have any semblance of intelligence, person and personality.
Stop it.
Eventually, they’ll try “I’m hurt that you won’t consider my suggestion for <advertised product/political position>.” And people will fall for it.
Being a curmudgeonly asshole is apparently the key to surviving in this brave new world and I’m way ahead of the game!
On the other hand, what an LLM is trying to autocomplete is a story of a conversation between its user and a helpful assistant, so maintaining the decorum of office politeness will align more with how it's fine-tuned and produce higher quality output.
Taking LLMs for what they are is in no way near to thinking these pieces of software possess personality, intelligence and reasoning capabilities.
You don't know. You strongly feel that you know. There's no standard procedure to assess the existence of phenomenal consciousness.
But with what I know of people and what I know of matrix multiplication, I have a strong suspicion.
A matmul (or a fixed finite sequence of matmuls) is not Turing-complete. An LLM that produces answer in the number of steps that is proportional to the length of the answer is not Turing-complete. An LLM with CoT that produces answer after a number of steps, which is determined by the LLM itself, is Turing-complete.
So, we do have a qualitative step between a matmul and modern LLMs. Of course, it doesn't guaranty having feelings, but it doesn't allow to simply dismiss the notion due to sheer computational inadequacy.
A machine is not turing complete, but by running it recursively on its own output, turing completeness emerges.
Who knows at what stage of intricacy consciousness emerges?
Arguably, either it does at some point, or it never does and we requiere the supernatural soul.
So, if it does, we can't disregard the possibility that we've reached it already.
Please.
You know it doesn’t experience joy, surprise or empathy too. You absolutely, undoubtedly know that.
First things first. You are wrong. I don't have a gut feeling that LLMs have no phenomenal states. Not anymore. I was skeptical that LLMs can form complex internal representations. Mechanistic interpretability has shown that they can. I'm still skeptical in regard to introspective mechanisms. One token being fed back into the network and KV cache update do seem lacking in comparison to the ubiquitous recurrent connections in the brain.
What I do know is that philosophy hasn't produced an operationalized definition of qualia. So, we need to approach the problem of LLMs' phenomenal states from other angles.
For example, we can argue that LLMs lack something, which seems to be relevant to the problem: a biological substrate, generality (non-CoT LLMs aren't Turing-complete), "something something feedback"[1], and things like that.
"Admit that you know that" is boring.
[1] I'm referring to https://www.astralcodexten.com/p/the-new-ai-consciousness-pa...
Reality is under no obligation to be fun. It might be boring but it is the truth.
The fact that you needed such effort to try and give this position any shred of sense only serves as proof of how nonsensical it is to do so.
The inability of people to reflect on their own beliefs amazes me to no end. How can you know that LLMs have no phenomenal states?
For example, I believe that rocks (ordinary rocks, not silicon chips) have no phenomenal states because for all we know they don't do any kind of information processing (relevant to self-reflection, at least) and I don't believe in panpsychism or dualism.
The more we demonize AI, the more likely we are to demonize humans.
[Which is not to say we can't criticize AI companies or worry about the many downsides to the technology. But anger and fear are not the answer either.]
At the risk of sounding like an LLM myself: this is exactly the key most seem to miss
For example, I can log in and have my computer work perfectly and treat it as a device. But the moment something goes wrong and I am frustrated and confused at something unexpected or if i get in range, I find myself getting angry at the computer often even reffering to it as a being.
So eventually yes, we do anthromorphize because we are human but we do it because its a very important way we model the world around us.
Of course all this can also be used against you depending on how much the creator of the product wants people to form a bond with it.
Furthermore yes it is pretty possible we stop being human when we stop anthromorphizing but that's not true as well. Research has found people on the autism spectrum usually do not anthromorphize and have a problem with empathy because they cannot model others well or their own selves. But that doesn't make them non humans.
How many humans would you damn to save that mechanical toad?
At the end of "Do Androids Dream of Electric Sheep?" [if I'm remembering correctly] Deckard finds a toad in the desert. He gives it to his wife(!) and she immediately bonds with it. Later they realize that it is mechanical (they find a battery compartment). But Deckard's wife still treats it like a real creature and wonders whether it will eat mechanical flies.
The core theme of "Do Androids Dream..." is that humans display empathy towards others, not because others deserve it, but because we are human. To be human is to empathize with other creatures, and when we stop empathizing, we stop being human.
This morning I was working with Codex and we found the solution to a really nasty bug. I was so happy/relieved/excited to have found it, and I shared my excitement with Codex. I know Codex is just a big matmul. I know it doesn't experience joy or surprise or empathy. But I still want it to behave excited, not because it is human, but because I am.