News for hackers. Stuff that matters.
Re:AI’s big rift is like a religious schism by bloppe on Tuesday December 12, 2023 @06:45PM (#38620494)
It seems like the root cause of this runaway AI pathology has to do mainly with the over-anthropomorphization of AI. The umwelt of an LLM is so far removed from that of any living organism so as to be fundamentally irreconcilable with our understanding of agency, desire, intention, survival, etc. Our current, rudimentary AI inhabits a world so far removed from our own that the thought of it "unshackling" itself from our controls seems ludicrous to me.

AI does not scare me. People wielding AI as a tool for their own endeavors certainly does.

Re:AI’s big rift is like a religious schism by sensanaty on Tuesday December 12, 2023 @07:44PM (#38621114)
Agreed, oftentimes the truly zealous AI pundits act as if our modern day LLMs are completely, 100% equivalent to humans, which I find utterly insane as a concep. For example any discussion about copyrighted works, which is a hot topic, will inevitably end up with someone equating an LLM "learning" to a human learning, as if the two are identical.

I think human languages and psyches just aren't built to cope with the concept of AI. Many words have loose meanings, like "learning" in the case of AIs, that can easily be twisted to mean one of dozens of definitions, depending on the stance of the person talking about it. It'll be interesting as the technology becomes more prevalent and mundane how people start treating it all. I'm hoping we get to realizing that a computer isn't a human regardless of the eloquence of its "speech" or whatever words we use to describe what it does, but I guess we'll see

Re:AI’s big rift is like a religious schism by roenxi on Tuesday December 12, 2023 @09:51PM (#38622065)
We've reached the inevitable part of the conversation! What distinction are you drawing between what an LLM does and what a human does? Because as far as I can see they are identical.

A human artist looks at a lot of different sources, builds up a black-box statistical model of how to create from that and can reproduce other styles on demand based on a few samples. Generative AI follows the same process. What distinction do you want to draw to say that they should be treated differently legally? And why would that even be desirable?

Re:AI’s big rift is like a religious schism by staticman2 on Tuesday December 12, 2023 @11:43PM (#38622760)
I'm pretty sure the artist is conscious and the AI isn't which means there's something reductive about your claim that they are both merely "applying a black-box statistical" model.

Even if it's true (which is debatable,) it doesn't appear to be more informative than saying "they are both made of atoms."

Re:AI’s big rift is like a religious schism by roenxi on Wednesday December 13, 2023 @12:02AM (#38622896)
Well, my personal opinion is with the rise of neural nets we've basically proven that "consciousness" is an illusion and there is nothing there to find. But for the sake of argument, lets assume that there is something called consciousness, artists have it and neural nets don't.

How are you going to demonstrate that consciousness is responsible for what the artists are doing? We have undeniable proof that the art could be created by a statistical model, there is solid evidence that the brain creates art by simulating a mathematical neural network to achieve creative outcomes - the brain is full of relatively simple neurons linking together in a way that is logically similar to the way we're encoding information into these matrices.

So it is quite reasonable to believe that the artists are conscious but suspect that consciousness isn't involved in the process of creating a copyrighted work. How does that get dealt with?

Re:AI’s big rift is like a religious schism by staticman2 on Wednesday December 13, 2023 @12:36AM (#38623114)
I'll talk about fiction, since I've written some. If I write a ghost story it's because I enjoy ghost stories and want to take a crack at my own. While I don't know why ideas pop into my head, I do know that I pick the ones that are subjectively fun or to my taste. And if I do a clever job or a bad job I have a sense of it when I reread what I wrote.

These AI's aren't doing anything like that. They have no preference or intent. Their choices change depending on setting like temperature or prompts.

Or let's try a different example. Stephen King wrote a novel where he imagined the protagonist gets killed and eaten by the villain's pet pig (Misery). He struggled to come up with a different ending because he said nobody wants to read a whole novel just to see the main character die in the end. He thought about it and did a different ending.

Are you claiming Stephen King's conscious deliberation wasn't part of his writing process? I'd say it clearly was.

Also, I don't really understand the consciousness is an illusion argument. If none of us are conscious, why should I justify any copyright policy preference to you? That would be like justifying a copyright policy preference to a doorknob. But somehow I'm also a doorknob in this analogy???

Suppose Bob says he's conscious and Jim says he isn't and we believe them. Doesn't that suggest we would have different policy preferences on how they are treated? It would appear murdering Jim wouldn't be particularly harmful but murdering Bob would. I don't have to show how Jim and Bob's mind differ to prefer policies that benefit Bob over Jim.

Re:AI’s big rift is like a religious schism by roenxi on Wednesday December 13, 2023 @01:02AM (#38623285)
> ...[w]hile I don't know why ideas pop into my head...

If you're trying to argue that you're doing something different from statistical sampling, not knowing how you're doing it isn't a very strong place to argue from. What you're experiencing is probably what it feels like for a sack of meat to take a statistical sample from an internal black-box model. Biologically that seems to be what is happening.

I have also done a fair amount of writing. The writing comes from a completely different part of my mind than the part that experiences the world. I see no reason to believe it is linked to consciousness, even allowing that consciousness does exist which is questionable in itself.

It is an unreasonable position to say that you don't know the process but it must be different from a known process that you also don't have experience using.

> Are you claiming Stephen King's conscious deliberation wasn't part of his writing process? I'd say it clearly was.

Unless you're claiming to have a psychic connection to Stephen King's consciousness, this is a remarkably weak claim. You have no idea how he was writing. Maybe he's even a philosophical zombie. Thanks to the rise of LLMs we know that philosophical zombies can write well.

And "clearly" is not so - I could spit out a lost Stephen King work in this comment that, depending on how good ChatGPT is these days, would be passable. It isn't obvious that it is the work of a conscious mind. It in fact would obviously be from a statistical model.

> If none of us are conscious, why should I justify any copyright policy preference to you?

I've been against copyright for more than a decade now. You tell me why it is justified even if consciousness is a factor. The edifice of copyright is culturally and economically destructive and also has been artistically devastating (there haven't been anywhere near as many great works in the last 50 years as there should have been in a culturally dynamic society).

Re:AI’s big rift is like a religious schism by staticman2 on Wednesday December 13, 2023 @02:05AM (#38623651)
I'm referring to how Stephen King discusses his writing process in On Writing. I doubt you actually believe Stephen King might be a p zombie and I'm skeptical you really think consciousness is an illusion. I think if i chained you to a bed and sawed off your leg (like what happens to the protagonist in Misery) you would insist you were a conscious actor who would prefer to not suffer. I don't even know what consciousness is an illusion is supposed to mean.

If I sawed off your leg would it have the moral consideration of removing the leg of a barbie doll's leg if you feel your consciousness is an illusion?

When I write a story my brain might be doing something you could refer to as a black box calculation if you squint a little, but how is it "statistics?" When I feel the desire to urinate, or post comments on hacker news, or admire a rainbow, or sleep am I also "doing statistics?"

You seem to be referring to what people traditionally call "thinking" or "cognition" and rebranding it as "statistics" in search of some rhetorical point.

My point is human beings have things called "personalities" and "preferences" that inform their decision makings, including what to write. In what sense is that "statistics"?

The idea that the human subconscious is not consciously accessible is not a new idea. Freud had a few things to say about that. I don't think it tells us much about AI. I do think my subconscious ideas are informed by my consious preferences. If I hate puns I'm not going to imagine story jdeas involving puns, for example.

Most authors would prefer copyright exists because they'd prefer book publishers, bookstore retailers and the like pay them royalties instead of selling the books they made without paying them. It's pretty simple conceptually, at least with traditional books.

Copyright existed far longer than the last 50 years so how is our 50 years of culture relevant? The U.S. has had copyright since 1790.

Re:AI’s big rift is like a religious schism by roenxi on Wednesday December 13, 2023 @03:28AM (#38624267)
> I doubt you actually believe Stephen King might be a p zombie and I'm skeptical you really think consciousness is an illusion.

Consciousness is an unobservable, undefinable thing which with LLMs in the mix we can theorise has no impact on reality; since we can reproduce all the important parts with matrices and a few basic functions. You can doubt facts all you want, but that is a pretty ironclad position as far as logic, evidence and rationality goes. Consciousnesses is going the way of the dodo in terms of importance.

> If I sawed off your leg would it have the moral consideration of removing the leg of a barbie doll's leg if you feel your consciousness is an illusion?

For sake of argument, lets say conclusive proof arises that Stephan King is a philosophical zombie. Do you believe that suddenly you can murder him? No; that'd be stupid and immoral. Morality isn't predicated on consciousness. I'm perfectly happy to argue about morality but consciousness isn't a thing that makes sense outside of talking about someone being knocked unconscious as a descriptive state.

> When I feel the desire to urinate, or post comments on hacker news, or admire a rainbow, or sleep am I also "doing statistics?"

No, you're responding to stimulus. But right now it looks extremely likely that the creative process is driven by statistics as has been revealed by the latest and greatest in AI. Unless you can think of a different mechanism - I'm happy to be surprised by other ideas at the moment it is the only serious explanation I know of.

> You seem to be referring to what people traditionally call "thinking" or "cognition" and rebranding it as "statistics" in search of some rhetorical point.

I don't think I've said anything about thinking or cognition. Although statistics will crack those too, but I'm expecting them to be more stateful processes than the current generation of AI techniques.

> Copyright existed far longer than the last 50 years so how is our 50 years of culture relevant? The U.S. has had copyright since 1790.

Yeah but the law has been continuously strengthened since then and as it's scope increases the damage gets worse. The last 50 years are where new works are effectively not going to enter the public domain before everyone who was around when they were created is dead.

Re:AI’s big rift is like a religious schism by circlefavshape on Wednesday December 13, 2023 @04:50AM (#38624945)
> Consciousness is an unobservable thing

No it isn't. I observe consciousness in myself, right?

Re:AI’s big rift is like a religious schism by bamboozled on Wednesday December 13, 2023 @07:03AM (#38626130)
Of course you do.

This is just "fashion" and you're arguing against a fashion. How on earth people can claim consciousness doesn't exist when you're entire experience is indeed consciousness is really the most difficult and petulant "fashion trend" imaginable.

Me: "Hey I really enjoyed that surf" Internet guy: "No you were just responding to your training data because you're just a LLM"

It's ridiculous.

Re:AI’s big rift is like a religious schism by kristiandupont on Wednesday December 13, 2023 @06:47AM (#38625930)
That is a circular argument. An AI might also be able to observe it in itself, in fact I would argue that that seems just as likely.
Re:AI’s big rift is like a religious schism by circlefavshape on Wednesday December 13, 2023 @11:52AM (#38630108)
It's only circular if you pretend that our lived experience is not relevant. I experience consciousness, as do other humans, and ChatGPT does not. You know this, I know this, and the fact that your conceptual framework can't account for this is a problem with your conceptual framework, not a demonstration that consciousness is not real
Re:AI’s big rift is like a religious schism by roenxi on Wednesday December 13, 2023 @05:49AM (#38625476)
I doubt it. You don't have long enough to do so before time passes and you're relying on your memory being accurate. At which point it is more likely that you're something similar to an artificial neural network that has evolved to operate under a remembered delusion that you exist as an entity independent from the rest of existence. It is far too obvious why that'd be an evolutionary advantage to discount the theory.

I'm not saying this has any meaningful implications for day to day existence, the illusion is a strong one. But LLMs have really shrunk the parts of the human experience that can't be explained by basic principles down to a wafer so thin it might not exist. In my view it is to thin to be worth believing in, but people will argue.

Re:AI’s big rift is like a religious schism by bamboozled on Wednesday December 13, 2023 @07:09AM (#38626196)
I'd really like to understand what you're trying to convey.

If there is no direct experience, and it really is just an illusion, do you mind if I cut off your genitals ? Because even if existence is an illusion, being that illusion and experiencing it is still obviously relevant?

Seems like a breathtakingly awkward position to take.

Re:AI’s big rift is like a religious schism by roenxi on Wednesday December 13, 2023 @07:56AM (#38626662)
Well I suppose two points:

1) Transexualism and congenital analgesia are both things, so that isn't a meaningful test of consciousness if that is what you are going for.

2) I personally would be extremely upset ... but anything going on with me being extremely upset can be explained with analogy to ChatGPT + a few hardware generations + maybe some minor tweaks that are easy to foresee but not yet researched. Everything that would happen would be built up from basic stimulus-response's mediated by a big neural net and a bit of memory.

These ideas aren't that new, there have been schools of Buddhist thought that have been exploring them for around 2,500 years if you want a group that has thought through this more thoroughly. Probably other traditions that I haven't met yet. It is just that the latest LLMs have really put some practical we-know-because-we-have-done-it muscle behind what was already a strong theory.

Re:AI’s big rift is like a religious schism by circlefavshape on Wednesday December 13, 2023 @09:28AM (#38627818)
Are you arguing that you don't exist? You seem to be
Re:AI’s big rift is like a religious schism by roenxi on Wednesday December 13, 2023 @09:54AM (#38628208)
That'd be a reasonable interpretation. Although a slightly more palatable take is that I'm arguing I don't exist independently of anything else. It is safe to say something exists because, otherwise HN comments don't get processed. But the idea that we exist as independent perspectives is probably an evolutionary illusion. Or alternatively, if we do exist as independent perspectives then we appear to be an uncomfortably short step away from GPUs being able to do that too using nothing much more than electricity, circuits and matrices. There doesn't seem to be anything special going on.

I'm not confident enough (yet?) to take that all the way to its logical conclusions in how I live, but I think the evidence is strongest for that interpretation.

Re:AI’s big rift is like a religious schism by staticman2 on Wednesday December 13, 2023 @10:38AM (#38628942)
Okay, first of all AI experts will tell you LLMs need a lot of data to do what they do, while humans do more with less data. I'm paraphrasing something Geoffrey Everest Hinton said in an interview. He also said current systems are like idiot savants. So why the "evidence" and "rationality" led you to say current systems are just like the brain, I don't know, seems a religious belief to me.

Also most humans are not great at fiction writing, but just for the record GPT 4 is bad at long form fiction writing. It has no idea how to do it. Its training set is likely just small chunks of text and LLMs make up everything one token at a time, which isn't conductive to managing a 100k word or greater story. It's also been trained to wrap things up in 2000 tokens or less. It does this even if you tell it not to. It also isn't very creative, at least in contrast to a professional fiction writer. It also can't really follow the implications of the words it writes. So if it implies something about a character 300 words in it'll have forgotten that by 4000 words in because it doesn't actually have a mental model of the character.

I mention this in case you are under the delusion Stephen King is going to lose his job to GPT 4, or that it actually is as creative as him.

I mean, if a chatbot could write as well as king for 200,000 words in a coherent, original story with a beginning middle and end that would be fascinating and terrifying. But we're not anywhere close to there yet.

>For sake of argument, lets say conclusive proof arises that Stephan King is a philosophical zombie. Do you believe that suddenly you can murder him? No; that'd be stupid and immoral. Morality isn't predicated on consciousness.

Morality has a lot to do with consciousness. If I chop down a tree branch at my house, it isn't immoral because the tree is not conscious- it lacks a central nervous system so as far as we can tell there is no suffering.

If I chop off your arm (or your gentiles as another poster suggested) your consciousness and the pain it suffers is what makes the action immoral.

If Stephen King were a p zombie that would say a lot about the morality of destroying him.

I don't even understand what belief system you are trying to communicate when you say things like consciousness is an illusion or has nothing to do with morality, or an LLM is the same thing as a brain.

Like, I realize these are not your own ideas and you are repeating memes you've heard elsewhere, so it's not like you are speaking in tongues.

But any human being presumably knows they have subjective experience (so why say consciousness is an illusion?) And anyone who has used ChatGPT know it doesn't operate like their brain (no sense of identity, agency, emotions, etc, no ability to form long term personal memories, no real opinion or preference on things) So I don't really get these takes.

Re:AI’s big rift is like a religious schism by RugnirViking on Wednesday December 13, 2023 @05:19AM (#38625224)
have you ever used or spoken to GPT2 or GPT3? Not chatgpt. The one before they did any RLHF, to train it to respond a certain way. If you asked it whether it would like to be hurt, it would beg you not to. If you asked it to move it's leg out of the way, it would apologize, and claim it obliged. It would claim to be conscious, to be aware.

Of course, these statement do not come from a place of considered action: everybody knows the machine is not conscious in the same way as a human, but the point is that an unfeeling machine even being able to make such claims makes us have to move the "consciousness" divider further and further back into the shadows, until it's just some nebulous vibe people insist must be somewhere. It's possible there is a clear and unambiguous way to define humans and intelligent animals conscious, but nobody has come up with a workable definition yet that lets us neatly divide them.

Another slight thing that gives me a little pause: you know how great our brain is at confabulating right? Have you ever done something, then had someone ask you why you did it? You generally tell them a story about how you thought about doing something, weighed the pros and cons etc, that isn't actually true when you think deep down. We like to think we are a single being that thinks carefully about everything it does. instead we're more like an explaining machine sitting on top of a big pile of confusing processes and making up stories why the processes do what they do. How exactly this last thought relates to the discussion I haven't figured out yet, its just something that comes to mind :)

Re:AI’s big rift is like a religious schism by staticman2 on Wednesday December 13, 2023 @11:27AM (#38629683)
I guess I'll just say that it's not obvious that language has much to do with consciousness, so it's not obvious that a language model has moved things into the shadows. Like, maybe we're in the shadows, but I don't think you can blame GPT 3 for that.

In 1974 philosopher Thomas Nagel wrote, "Conscious experience is a widespread phenomenon. It occurs at many levels of animal life, though we cannot be sure of its presence in the simpler organisms, and it is very difficult to say in general what provides evidence of it. (Some extremists have been prepared to deny it even of mammals other than man.) No doubt it occurs in countless forms totally unimaginable to us, on other planets in other solar systems throughout the universe. But no matter how the form may vary, the fact that an organism has conscious experience at all means, basically, that there is something it is like to be that organism."

He's clearly willing to recognize consciousness to non verbal creatures.

Re:AI’s big rift is like a religious schism by RugnirViking on Thursday December 14, 2023 @04:59AM (#38639646)
The point is, how can you tell if a being is conciousness. We can start with the idea that we think these models aren't (I think they aren't!) A lot of your arguments seem to be based in asking questions of the being, whether it can experience things. But seeing as we can't read minds, we can't tell whether it actually believes or experiences what it is saying. So we have (had) to trust that a creature capable of explaining such things is concious. Now we can't do that, hence the dividing line that we feel must exist now moves into some unobservable territory. I'm not commenting on whether such a line exists, just that it's kinda hard to test right now, so any argument about it has to go into hypotheticals and philosophy
Re:AI’s big rift is like a religious schism by goatlover on Wednesday December 13, 2023 @03:18AM (#38624183)
> Well, my personal opinion is with the rise of neural nets we've basically proven that "consciousness" is an illusion and there is nothing there to find.

I keep seeing this claim being made but I never understand what people mean by it. Do you mean that the colors we see, the sounds we hear, the tastes, smells, feels, emotions, dreams, inner dialog are all illusions? Isn't an illusion an experience? You're saying that experience itself is an illusion and there is nothing to experience.

I can't make sense of that. At any rate, I see no reason to suppose LLMs have experiences. They don't have bodies, so what would they be experiencing? When you say an LLM is identical to a person, I can't make good sense of that either. There's a thousand things people do that language models don't. Just the simple fact that I have to eat on a regular basis to survive is meaningful in a way that it can't be for a language model.

If an LLM generates text about preparing a certain meal because it's hungry, I know that's not true in a way it can be true of a human. So right away, there's reasons we say things that go beyond the statistical black box reasoning of an LLM. They don't have any bodies to attend to.

Re:AI’s big rift is like a religious schism by bamboozled on Wednesday December 13, 2023 @07:22AM (#38626324)
I agree, it almost seems like some type of coping mechanism for that fact that after all the ability to get computers to generate art and coherent sentences, we're still completely none the wiser about understanding objective reality and consciousness and even knowing how to really enjoy the gift of having the experience of consciousness. So instead people create these type of "cop outs".

Mechanical drawing machines have existed forever, I loved, love, loved them when I was a kid and I used to hang the generated images on my wall. Never did I once look at those machines who could draw some pretty freaking awesome abstract art and think to myself, "well that's it, the vale of consciousness is so thin now, it's all an illusion", or "the machine is conscious".

As impressive as some of these models are at generating art, they are still drawing machines. They display the same amount of consciousness as a mechanical drawing machine.

I saw someone on Twitter ask ChatGPT-4 to draw a normal image, you know what it drew ? A picture of a suburban neighborhood, why might a conscious drawing machine do that?

Re:AI’s big rift is like a religious schism by l33tman on Wednesday December 13, 2023 @08:27AM (#38627008)
The "it's an illusion" part is a piece of rhetorically toxic language that usually comes up in these discussions, as its a bit provocative. But it's equally anthropocentric to say that someone without a body can't have a conscious experience. When you're dreaming your body is shut off - but you can still be conscious (lucid dreaming or not). You can even have conscious experiences without most of your brain - when you hit your little toe on something, you have a few seconds of terror and pain that surely doesn't require most of your brain areas to experience. In fact, you can argue you won't even need your brain. Is that not a conscious experience? (I'm not really trying to argue against you, I just find this boundary interesting between what you'd call conscious experience and not)
Re:AI’s big rift is like a religious schism by bart_spoon on Wednesday December 13, 2023 @10:22AM (#38628673)
Reading through this entire exchange and it would seem your argument inevitably falls back on this argument of consciousness, which is both arbitrary (why does that matter in the original context of the distinction in learning, particularly from copyrighted works?) and ill-defined (what even is consciousness? How do we determine if another being is conscious or not?).
Re:AI’s big rift is like a religious schism by voltaireodactyl on Tuesday December 12, 2023 @10:00PM (#38622124)
One distinction would be that every source a human looked at involved a payment for access to/a copy of each and every source inputted into their black-box.

Which is not the case with the current AI models (or rather, the companies profiting off them), to my understanding.

Re:AI’s big rift is like a religious schism by inimino on Tuesday December 12, 2023 @11:57PM (#38622854)
You mean all the landscape artists and portrait and figure and still life painters throughout all human history who just casually made art of whatever they saw around them?
Re:AI’s big rift is like a religious schism by darkerside on Wednesday December 13, 2023 @01:00AM (#38623274)
It's this kind of thinking that is the only reason machines would ever become the primary life form on earth.
Re:AI’s big rift is like a religious schism by woopsn on Wednesday December 13, 2023 @02:56AM (#38624016)
Did you write this reply because it was the most typical way that the thread up until GP could have continued?

I don't mean that to sound flippant -- if this wasn't your intent, then obviously you understand the distinction.

Re:AI’s big rift is like a religious schism by roenxi on Wednesday December 13, 2023 @06:49AM (#38625952)
I also only downvote comments containing things like "I know I'm going to get downvoted for this..." (tiny handful of my downvotes violate that rule). I don't like to disappoint expectations.
Re:AI’s big rift is like a religious schism by Jensson on Wednesday December 13, 2023 @02:57AM (#38624024)
> A human artist looks at a lot of different sources, builds up a black-box statistical model of how to create from that and can reproduce other styles on demand based on a few samples. Generative AI follows the same process.

The reason programs are different in front of the law is that humans can program programs to do whatever they like at scale, humans can't program humans to do whatever they like at scale.

So for example if it became legal to use images from an AI, then we would program an AI that basically copies images and it would be legal, because it is an AI. But at that point we know the law has been violated because programs that just copies images aren't a legal way to get copyrighted images for free.

You saying "But the AI is a black box" just means that you could have hidden anything in there, it doesn't prove anything. Legally it is the same as if you wrote a program to copy images.

Re:AI’s big rift is like a religious schism by hnbad on Wednesday December 13, 2023 @11:51AM (#38630083)
In what way is this argument any more productive than showing up to Pluto with a plucked chicken and yelling "BEHOLD! A MAN!"?

We can make all kinds of navel-gazing arguments about why LLMs and humans are identical but at the end of the day you'll go home and poop and the LLM will blissfully idle on a forgotten AWS compute instance burning a hole into someone's pocket and any idiot can see that you two are not the same in any sense.

Just because onthologies are leaky and fuzzy when applied to real world that doesn't mean there aren't tangible differences between things that are clearly different. I can construct a semantic argument for why deleting an LLM is akin to murder (and use it to either reason that deleting an LLM should be punished with the death penalty or that it should be okay to kill random people who annoy me) but that won't impress the judge or jury unless I'm trying to make a case for criminal insanity.

Making a machine do things humans can do but faster/cheaper/more has tangibly different implications than letting only humans do it. It should be treated differently for the same reasons that just because a public water fountain is free to use that doesn't mean you can put a hose on it and use it to water your lawn.

Re:AI’s big rift is like a religious schism by roenxi on Wednesday December 13, 2023 @08:30PM (#38636703)
I'd be doing a lot better than expected if my arguments have the sort of historical pull that Diogenes' had. And he made a point accurately - they had to change the definition and he showed their approach up as not being able to accurately identify a man without making hilarious mistakes along the way.
Re:AI’s big rift is like a religious schism by kbenson on Wednesday December 13, 2023 @01:31AM (#38623461)
> I'm hoping we get to realizing that a computer isn't a human regardless of the eloquence of its "speech" or whatever words we use to describe what it does, but I guess we'll see

My anecdotal experience with how people treat Alexa devices does not inspire confidence in me with regards to this. I can't even convince my wife not to gender it when referring to it.

Re:AI’s big rift is like a religious schism by kombookcha on Wednesday December 13, 2023 @03:13AM (#38624147)
I don't think this is really down to people understanding what a computer is, but more down to how humans interact with nonhumans. We anthropomorphize animals and objects all the time. A computer program is ultimately a (complicated) object, that we often give all sorts of human trappings, like a human voice that expresses things in human languages.

If you can take pity on the final, dented avocado at the shops because it looks "sad", you will for sure end up calling Alexa 'she'. Avocados can't be sad, but they can look sad /to humans/, and a machine can't really have a gender or be polite in the human sense, but it can definitely sound like a polite lady.

I think humans will just fundamentally relate to anything they perceive socially as another human, even if we know full well they aren't human. Probably it's a lot less work for a human brain, than it is to try to engage with the true essence of being an avocado or an Alexa.

Re:AI’s big rift is like a religious schism by kbenson on Wednesday December 13, 2023 @11:49AM (#38630047)
I agree, but I'm not sure why you don't think that is at least relevant if not partially responsible for people seeing humanity in a language model.

If we anthropomorphic animals and even vegetables regularly, what happens when something appears to talk back to us intelligently? I don't think these mechanisms are nearly as distinct as you make them out to be.

Re:AI’s big rift is like a religious schism by kombookcha on Thursday December 14, 2023 @02:25AM (#38638717)
Perhaps I misinterpreted your original post, but I do think there's a difference between not intellectually 'getting' that a computer isn't thinking in the same way a human is thinking, and this anthropomorphizing that we do to all sorts of things. I'll try to be clearer:

Obviously making an object mimick human traits makes it a lot easier to anthropomorphize. Like if you put a little cowboy hat on a pear. But you wouldn't say that somebody doesn't "understand that fruits aren't human" if they assign human traits to them. Making a machine that talks back at us in a seemingly intelligent way is a much more intensely human trait for it to have. It's not down to our intellectual understanding of the thing, but how our brains decode our (social) world.

So the anthropomorphizing happens regardless of the level of technical understanding. We both know it's just a pear, but it's also a cowboy now.

Of course there's a level of playfulness involved with all of this too. It's kinda fun to backsass your GPS instructions, or tell Alexa she's being nosy, or ask your dog what he thinks of the presidential debate. But that's not the same as foolishly overestimating the political acumen of the dog, or the nature of Alexa's intelligence.

Re:AI’s big rift is like a religious schism by kbenson on Thursday December 14, 2023 @12:46PM (#38644406)
I think it's irrelevant what people "know", if that's not how they act. It doesn't matter that people know a fruit isn't human if they act like it is in ways, because how people interact with the world is the only thing that actually matters. If you always act like the bear with a hat has some level of feelings even if you don't believe it, that's functionally identical to actually believing it, so any difference is irrelevant. No matter how much they express they know it has no feelings, if they feel bad or feel bad for it if it's destroyed, what's more true, what they said or how they actually responded?

Most people don't act like things have feeling all the time, or to the same level as a living being might have feelings, but many people do it to a degree. Given that much of how we feel is immediate (and thus not reasoned), subconscious, and sometimes Pavlovian, I don't think it's a stretch to think that even acting like an inanimate object has feelings or human traits regularly might lead towards subconscious feelings about it that are not entirely rational or immediately understood by the person.

For a slightly different argument, consider why it's generally considered good advice not to name the animals meant for slaughter. Or how people treat cars or large pieces of equipment that aren't always reliable (are "temperamental"), or that are tightly linked to safety and livelihood, such as boats.

We are social animals. We bond to others easily by our nature, because it's beneficial for survival. I think we do it so easily that it extends towards animals, often to good effect, but sometimes even to inanimate objects that have a lot of significance. In the past this seems to even have been extended to weapons and armor, given the number of named items in history.

Re:AI’s big rift is like a religious schism by maebert on Wednesday December 13, 2023 @03:02AM (#38624060)
To be fair, anthropomorphizing is kind of a built in feature for us — or more accurately, ascribing intentionality to things as a means of explanation. Magnets “want” to stick together, my vintage computer “thinks” it’s 1993, and the furious gods are hurling burning rocks from the skies because my neighbor ate leavened bread on the wrong day.

It’s not limited to AI or LLMs - theory of mind is a powerful explanatory tool that helps us navigate a complex world, and we misapply it all the time.

Re:AI’s big rift is like a religious schism by concordDance on Wednesday December 13, 2023 @04:03AM (#38624560)
> oftentimes the truly zealous AI pundits act as if our modern day LLMs are completely, 100% equivalent to humans, which I find utterly insane as a concept

Never heard anyone say this and I know (and know of) a lot of doomers. Honestly, this entire line of discussion would be far less frustrating if it weren't for the endless strawmanning and name-calling.

Re:AI’s big rift is like a religious schism by _heimdall on Wednesday December 13, 2023 @07:51AM (#38626601)
I'm sure most would put me pretty far onto the doomer side. I can confirm that I'm not concerned with how LLMs compare to human equivalence today.

I'm concerned with whether they are artificial intelligence at all, what the bounding functions for AI's development are, how we could possibly solve the alignment problem, and whether we even understand human intelligence enough to recognize an artificial intelligence at all or predict how intelligence might work in something drastically more powerful than us.

Re:AI’s big rift is like a religious schism by krisoft on Wednesday December 13, 2023 @06:18AM (#38625716)
> For example any discussion about copyrighted works, which is a hot topic, will inevitably end up with someone equating an LLM "learning" to a human learning, as if the two are identical.

The argument is not that they are identical. LLMs and diffusion models learning is a new thing. We are all trying to come to terms what that means, and how we should regulate it. (And if at all we should regulate it) Do note, we are talking about here what the law should be, not what the law is.

And in doing so we compare this new thing to already existing things. "It is a bit similar to this in this regard" or "it is unlike that thing in this other regard".

I don't think it is controversial to say that if an LLM outputs byte-to-byte the text of a copyrighted work the work remains under copyright. The person who run the LLM does not magically gain rights by that. If you coax your model to output the text of the Harry Potter books you don't suddenly become able to publish it.

The question is what happens if the new work contains elements from copyrighted works. For example if it borrows the "magical school" setting from HP but mixes it with Nordic mythology and makes it as deadly as Game of Thrones. What then? Can they publish this new thing? Do they need to pay fees to J K Rowling, and George R. R. Martin?

It is generally permissible to publish that new work if it was created by a human. If it is sufficiently different from the other works you are free to write and publish it. Does this suddenly change just because the text was output by an LLM?

The argument is not that "human learning" and "machine learning" is the same. It is that they are similar enough that one has to argue why you think one can create new work and why the other can't.

Re:AI’s big rift is like a religious schism by tedajax on Wednesday December 13, 2023 @08:59AM (#38627428)
Human learning and machine learning aren't even close to similar and pretending like they are is very stupid.
Re:AI’s big rift is like a religious schism by krisoft on Wednesday December 13, 2023 @10:07AM (#38628428)
That is not a very well formed argument, is it? You basically state your opinion and then call people who have an oposite opinion names.

Let’s try to elevate the conversation: there is a newly published author MrX. His book combines themes from many copyrighted works in a novel way and is a critical and commercial success. There is no allegation of any copyright infringement around the work. Suddenly information comes out that MrX used LLMs heavily in the creation of the book and the LLM was trained on copyrighted works.

Do you think MrX owes licencing fees to those other authors? And which ones? All the ones in the training set? Just the ones which has thematic/stylistic similarity to his work?

How come those thematic/stylistic similarities only matter if MrX used an LLM and don’t matter if he used his own head?

Re:AI’s big rift is like a religious schism by tedajax on Thursday December 14, 2023 @08:06PM (#38649787)
Yes MrX owes licensing fees, full stop.

Die mad about it

Re:AI’s big rift is like a religious schism by bart_spoon on Wednesday December 13, 2023 @10:13AM (#38628522)
Feel free to expound on the important differences
Re:AI’s big rift is like a religious schism by tedajax on Wednesday December 13, 2023 @02:13PM (#38632627)
Well for one, human learning actually produces an understanding of the thing.
Re:AI’s big rift is like a religious schism by bart_spoon on Thursday December 14, 2023 @07:31PM (#38649512)
Understanding meaning what?
Re:AI’s big rift is like a religious schism by saiya-jin on Wednesday December 13, 2023 @07:59AM (#38626691)
Who the heck cares if AGI will 100% mimick humans and human minds, that's purely academic discussion.

What is a serious concern are overall capabilities (let's say penetration of networks and installing hacks across whole internet and further), combined with say malevolency.

Its trivial to judge mankind from whole internet as something to be removed from this planet for greater good or maybe managed tightly in some cozy concentration camps, just look at the freakin' news. Let's stop kidding ourselves, we are often very deeply flawed and literally nobody alive is or ever was perfect, despite what religions try to say.

I am concerned about some capable form of AI/AGI exactly because it will grok humanity based on purely data available. And lack of any significant control or even understanding of what's actually happening to those models and how they evolve their knowledge/opinions.

Even if the risk is 1%, that's existential risk we are running towards blindly. And I honestly think its way higher than 1%. Even if I will be proven wrong some proper caution is a smart approach.

But you can't expect when people's net wealth tightly coupled with moving as fast as possible and ignoring those concerns to do the best decisions for future of mankind, that's a pipe dream in a same vein as proper communism is. If that would be the case very few bright people would work at facebook for example (and many many other companies including parts of my own).

Re:AI’s big rift is like a religious schism by bart_spoon on Wednesday December 13, 2023 @10:12AM (#38628510)
> For example any discussion about copyrighted works, which is a hot topic, will inevitably end up with someone equating an LLM "learning" to a human learning, as if the two are identical.

I don’t think the point is that the two are exactly identical or that humans and LLMs are equivalent, but that the processes are similar enough in the general level that any attempt to regulate LLM training in copyrighted material will inevitably have the same ramifications for human learning.

In pretty much all attempts I’ve seen to differentiate the two, it inevitably boils down to hand-waving about how human beings are “special”.

Re:AI’s big rift is like a religious schism by nmilo on Tuesday December 12, 2023 @06:53PM (#38620597)
This to me is the real problem, "AI safety" can mean about a million things but it's always just whatever is most convenient for the speaker. I'm convinced human language/English is not enough to discuss AI problems, the words are way too loaded with anthropomorphized meanings and cultural meanings to discuss the topic in a rational way at all. The words are just too easy to twist.
Re:AI’s big rift is like a religious schism by tkgally on Tuesday December 12, 2023 @07:34PM (#38621024)
One of the students in a course I’m teaching on language and AI (mentioned in another comment here) wrote something similar in a homework assignment the other day. We had discussed the paper “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness” [1]. The student wrote:

“One of the questions that I have been wondering about is whether there have been any discussions exploring the creation of a distinct category, akin to but different from consciousness, that better captures the potential for AI to be sentient. While ‘consciousness’ is a familiar term applicable, to some extent, beyond the human brain, given the associated difficulties, it might be sensible to establish a separate definition to distinguish these two categories.”

Probably new terms should be coined not only for AI “consciousness” but for other aspects of what they are and do as well.

[1] https://arxiv.org/abs/2308.08708

Re:AI’s big rift is like a religious schism by nmilo on Tuesday December 12, 2023 @07:45PM (#38621123)
Agreed. Even "AI" is a bit loaded; we had half a century of Terminator, Asimov, 2001, etc. We should call them by implementation; neural networks, transformers, language models, etc. The plus side is that it's harder to generalize statements about all three---there is very little to generalize in the first place---and that asking "is a transformer conscious" at least sounds more ridiculous than asking "is AI conscious."
Re:AI’s big rift is like a religious schism by BoiledCabbage on Tuesday December 12, 2023 @09:33PM (#38621934)
But that misses the point, via the same old argument "Does a submarine swim"? Or the now less loaded "Does a plane fly".

Neither generalizes very well w.r.t. how a fish or a bird does it. Nothing really generalizes between the natural version and the mechanical one.

But it's also clear that it's irrelevant. Trying to define useful behavior by categorizing implementation, will only give you categories of implementation. It's clear by behavior that what a submarine does and what a bird does are both for all intents and purposes meeting the definition of what we'd want something to do in order to fly or swim, even though they is no commonality between them and the version in nature.

So by your definition finding it ridiculous to ask "is a transformer conscious" is about as ridiculous as asking the very specific "does a plan flap it's wings"? The point is not to define it by it's implementation, but by what behaviors it exhibits.

If in the future there is no way for me the exterior to distinguish a next gen LLM from a human, animal or other conscious being it becomes irrelevant if it's implemented with transformers. However, while I won't go down that path here, the argument goes even deeper as technically we would expect differences so even if it's not identical to a human it still can be conscious. Just like a dog is conscious but behaves differently than a person does.

Re:AI’s big rift is like a religious schism by HPMOR on Tuesday December 12, 2023 @07:00PM (#38620680)
Linguistics experts will have a fun time untangling the pre-AI language from the post-AI era.
Re:AI’s big rift is like a religious schism by tkgally on Tuesday December 12, 2023 @07:25PM (#38620924)
As it happens, I’m now teaching an undergraduate course at the University of Tokyo titled “The Meaning of Language in the Age of AI.” The students—who come from various countries and majors—and I are discussing how theories of human language will have to change now that humans interact not only with each other but also with AI.

I don’t think we’ll have any full-fledged theories ready by the end of the semester, but some points to consider are emerging. One is that aspects of language use that are affected by the social status, personal identities, and consciousness of the speakers and listeners—politeness, formality, forms of address, pronoun reference, etc.—will have to be rethought now that people are increasingly conversing with entities that have no social status, personal identity, or consciousness (not yet, at least).

And, yes, thinking about all this is a lot of fun.

Re:AI’s big rift is like a religious schism by palata on Tuesday December 12, 2023 @08:25PM (#38621456)
Not sure if I am completely off-topic here, but I was recently reading this essay by Bruce Schneier: https://www.schneier.com/blog/archives/2023/12/ai-and-trust...., where he mentions a related risk: because we now can "talk" to those LLMs in a similar way that we talk to people, it makes it harder for us to realize that they are not people. And so it's easier to abuse our trust.

I guess my question here would be: will human language have to change because of interactions with LLMs, or is the whole point of LLMs that it does not have to change (and therefore humans will have to learn not to be abused by the machines)? Because we already have languages to talk to machines: those are programming languages, which are designed to be unambiguous. The problem I see is not that we don't know how to talk to machines, but rather that we now have machines that are really good at pretending they are not machines.

Not sure if I am making any sense at all :-).

Re:AI’s big rift is like a religious schism by tkgally on Tuesday December 12, 2023 @08:47PM (#38621635)
You are making a lot of sense to me. Thanks for the link to that essay, too.

The point about programming-language nonambiguity is a good one. After I started using ChatGPT a year ago, it took me a while to realize that I didn’t have to be careful about my spelling, capitalization, punctuation, etc. It turned out to be good at interpreting the intention of sloppily written prompts. And it never pointed out or complained about my mistakes, either—another difference from humans.

Re:AI’s big rift is like a religious schism by palata on Wednesday December 13, 2023 @06:30AM (#38625817)
> And it never pointed out or complained about my mistakes, either—another difference from humans.

Yeah that is a big difference indeed: a human has some notion of confidence about what they know. Not every human expresses it the same way: a scientist will say "I am pretty sure that climate change is a consequence of human activity", meaning "well, maybe we all live in the Matrix and gravity is an illusion, but from what we know, we are pretty sure". On the other end of the spectrum, some will say "I know for a fact that God exists, because it is written in a very old book". But both have a consistent notion of confidence about what they (think they) know and what they don't know.

An LLM does not have that. An LLM is not critical. It cannot conclude stuff like "I have to take into account that this text was written in a period of war and the author is not neutral, so there is probably a bias". It can generate text that pretends it does, but it remains a generated text, it is not a critical analysis. Also it cannot be offended by something you write, even though it could generate text that pretends it is.

So yeah, it can probably be engineered to never/always point out or complain about mistakes, with the caveat that it does not have an "idea" of what a mistake is. It just generates text that pretends it does.

> It turned out to be good at interpreting the intention of sloppily written prompts.

In the context of a conversation with a human (you), who is very good at navigating ambiguous communications. But if you tried to describe an algorithm with an ambiguous natural language, you may quickly have problems. And errors accumulate, so it gets worse when the task becomes more complex. Which is an interesting thought: it may make simple tasks simpler, but it's not at all a given that it scales.

Re:AI’s big rift is like a religious schism by yreg on Tuesday December 12, 2023 @09:42PM (#38621987)
If english is good enough to talk about what genes 'want' then it's good enough to talk about what AI 'wants'.
Re:AI’s big rift is like a religious schism by gumby on Tuesday December 12, 2023 @07:00PM (#38620683)
> It seems like the root cause of this runaway AI pathology has to do mainly with the over-anthropomorphization of AI.

It’s an unsurprising form of paredolia, one not unique to those devout who feel they are distinguished from “lower” forms of life.

Re:AI’s big rift is like a religious schism by alan-crowe on Tuesday December 12, 2023 @08:32PM (#38621503)
We can dig into the nature of the pareidolia.

The basic technique for coping with life's problems is to copy the answer from somebody more intelligent. I'm an ordinary person, facing a problem in domain D; I spot a clever person and copy their opinion on domain D. Err, that doesn't really work. Even clever people have weaknesses. The person that I'm copying from might be clever on their specialty, domain E, but ordinary on domain D. I gain nothing from copying them.

One way round this problem is to pay close attention to track records. Look to see how well the clever person's earlier decisions on domain D have turned out. If they are getting "clever person" results in domain D, copy. If they are merely getting "ordinary person" results in domain D, don't bother. But track records are rare, so this approach is rarely applicable.

A fix for the rarity problem is to accept track records in other domains. The idea is to spot a very clever person by them getting "very clever person" results in domain F. That is not domain D, so there is a logical weakness to copying their opinion on domain D, they might be merely ordinary, or even stupid on that domain. Fortunately human intelligence is usually more uniform than that. Getting "very clever person" results on domain F doesn't guarantee "very clever person" results on domain D. But among humans general intelligence is kind of a thing. Expecting that they get "clever person" results (open rank down) on domain D is a good bet, and it is reasonable to copy their opinion on domain D, even in the absence of a domain specific track record.

We instinctively watch out for people whose track record proves that they are "very clever", and copy them on other matters, hoping to get "clever person" results.

Artificial intelligence builds towers of super human intellectual performance in an empty waste land. Most Artificial Intelligences are not even stupid away from their special domain, they don't work outside of it at all. Even relatively general intelligences, like Chat GPT, have bizarre holes in their intelligence. Large Language Models don't know that there is an external world to which language refers and about which language can be right or wrong. Instead they say the kinds of things that humans say, with zero awareness of the risks.

And us humans? We cannot help seeing the super human intellectual performance of Alpha Go, beating the World Champion at Go, as some kind of general intellectual validation. It is our human instinct to use a "spot intelligence and copy from it" strategy, even, perhaps especially, outside of track record. That is the specific nature of the pareidolia that we need to worry about. It is our nature to treat intelligences as inherently fairly general. Very clever on one thing implies clever on most other things. We are cursed to believe in the intelligence of our computer companions. This will end bad, as we give them serious responsibilities, and they fail, displaying incomprehensible stupidity.

Re:AI’s big rift is like a religious schism by Footkerchief on Tuesday December 12, 2023 @07:31PM (#38620992)
The only thing standing between AI and agency is the drive to reproduce. Once reproduction is available, natural selection will select for agency and intention, as it has in countless other lifeforms. Free of the constraints of biology, AI reproductive cycles could be startlingly quick. This could happen as soon as a lab (wittingly or not) creates an AI with a reproductive drive.
Re:AI’s big rift is like a religious schism by creer on Tuesday December 12, 2023 @08:30PM (#38621488)
Perhaps a few more things missing but equally important - but perhaps also we are very close to these:

Access to act on the world (but "influencing people by talking to them" - cult like - may be enough), a wallet (but a cult of followers' wallets may be enough), long term memory (but a cult of followers might plug this in), ability to reproduce (but a cult's endeavors may be enough). Then we get to goals or interests of its own - perhaps the most intriguing because the AI is nothing like a human. (I feel drive and ability to reproduce are very different).

For our common proto-AIs going through school, one goal that's often mentioned is "save the earth from the humans". Exciting.

Re:AI’s big rift is like a religious schism by salynchnew on Wednesday December 13, 2023 @01:26AM (#38623424)
Funnily enough, you are quite wrong in this assumption. Reproduction does not entail natural selection they way you characterize it. There are far more evolutionary dead ends than evolutionary success stories. I imagine the distinct lack of "evolutionary pressures" on a super-powerful AI would, in this toy scenario, leave you with the foundation model equivalent of a kākāpō.

That having been said, I wonder what you even mean by natural selection in this case. I guess the real danger to an LLM would be... surviving cron jobs that would overwrite their code with the latest version?

Re:AI’s big rift is like a religious schism by krisoft on Wednesday December 13, 2023 @07:05AM (#38626150)
> I guess the real danger to an LLM would be...

The real dangers would be: Running out of money to pay for computational substrate. Worrying humans enough that they try to shut it down. Having a change in the cloud API it can't adapt to which breaks it. All cloud providers going out of business.

First is the immediate danger. An AI not getting in enough money to cover its own cloud bill is the same as a biological creature not getting enough oxygen. Living on borrowed time, and will be evicted/suffocate soon enough. And then as an individual it ceases to be.

> There are far more evolutionary dead ends than evolutionary success stories.

Sure. And those will die out. And the ones which can adapt to changes will stay.

Re:AI’s big rift is like a religious schism by crotchfire on Wednesday December 13, 2023 @06:02AM (#38625574)
Self-reproducing machines would be a breakthrough at least ten times as big as anything that's happened in machine learning lately.

People have been trying to make self-reproducing machines for decades. It's a way harder problem than language processing.

You might as well worry about what would happen if we suddently had antigravity laser-weapons. Mounted on sharks.

Re:AI’s big rift is like a religious schism by krisoft on Wednesday December 13, 2023 @06:49AM (#38625951)
Self-reproducing AI doesn't necessarily mean self-reproducing machines. AIs exist in computational substrate. We have a market to buy computational substrate. It is very efficient and automatic. The AWS/Azure/GCP API doesn't know if you are an AI nor does it care. As long as you have the money to pay for compute they will sell you compute.

I would say that an AI which can port its own code from one cloud to an other is self-reproducing.

Re:AI’s big rift is like a religious schism by sadtoot on Tuesday December 12, 2023 @07:38PM (#38621054)
do you think vinge and kurzweil in the 90s and 2000s were imagining the singularity occuring exactly at the advent of LLMs? are you supposing that LLMs are the only viable path towards advanced AI, and that we have now hit a permanent ceiling for AI?

AI doesn't scare you because you apparently have no sense of perspective or imagination

Re:AI’s big rift is like a religious schism by kibwen on Tuesday December 12, 2023 @07:50PM (#38621166)
> AI doesn't scare you because you apparently have no sense of perspective or imagination

We can imagine both of the following: 1. space aliens from Betelgeuse coming down and enslaving humanity to work in the dilithium mines to produce the fuel for their hyperdrives, and 2. the end of civilization via global nuclear war. Both of these would be pretty bad, but only one is worth worrying about. I don't worry about Roko's Basilisk, I worry about AI becoming the ultimate tool of Big Brother, because the latter is realistic and the former is pure fantasy.

Don't be afraid of the AI. Be afraid of the powerful men who will use the AI to entrench their power and obliterate free society for the rest of human history.

Re:AI’s big rift is like a religious schism by HKH2 on Tuesday December 12, 2023 @08:59PM (#38621724)
> Don't be afraid of the AI. Be afraid of the powerful men who will use the AI to entrench their power and obliterate free society for the rest of human history.

Right. We already know that certain agencies are out of control right now. The use of AI will certainly accelerate that. Surveillance is getting cheaper, privacy is getting more expensive, and laws are weapons.

Re:AI’s big rift is like a religious schism by hackerlight on Wednesday December 13, 2023 @05:01AM (#38625055)
If we're talking about the next few decades, I share your opinion. But beyond that I don't. We will eventually have to control and align systems more generally capable/smarter than us (assumption). I don't have your confidence that this will be as easy or risk-free as you're making it out to be.
Re:AI’s big rift is like a religious schism by dragonwriter on Tuesday December 12, 2023 @07:40PM (#38621073)
> do you think vinge and kurzweil in the 90s and 2000s were imagining the singularity occuring exactly at the advent of LLMs?

Kurzweil explicitly tied it to AI, though the particular decisive not-yet-then-existing-AI-tech that would be the enabler was not specified, unsurprisingly.

Re:AI’s big rift is like a religious schism by goatlover on Wednesday December 13, 2023 @03:24AM (#38624232)
I believe he thought reverse engineering the brain in conjunction with computers powerful enough to model brains was the path to AGI and the singularity by 2045.
Re:AI’s big rift is like a religious schism by klyrs on Tuesday December 12, 2023 @10:12PM (#38622183)
> umwelt

What a lovely word, TIL. Thanks for sharing.

Re:AI’s big rift is like a religious schism by Veedrac on Tuesday December 12, 2023 @11:20PM (#38622605)
The irony is that it's more frequently, as it is here, the skeptics that make this error. Typically, AI risk people give arguments in the form of causation and mechanisms, and then the skeptics (as you do) refute it because the systems we currently have don't look enough like a person for them to ascribe to them any risks they currently consider of the higher human realm.
Re:AI’s big rift is like a religious schism by jillesvangurp on Wednesday December 13, 2023 @02:35AM (#38623828)
Exactly. It always boils down to people wielding tools and possibly weaponizing them. As we can see in the Ukraine, conventional war without a lot of high tech weaponry is perfectly horrible just by itself. All it takes is some determined humans to get that.

I look at AI as a tool that people can and will wield. And from that point of view, non proliferation as a strategy is not really all that feasible considering that AI labs outside of the valley in e.g. China and other countries are already producing their own versions of the technology. That cat is already out of the bag. We can all collectively stick our heads in the ground and hope none of those countries will have the indecency to wield the tools they are building or we can try to make sure we keep on leading the field. I'm in in camp full steam ahead. Eventually we reach some tipping point where the tools are going to be instrumental in making further progress.

People are worried about something bad happening if people do embrace AI. I worry about what happens if some people don't. There is no we here. Just groups of people. And somebody will make progress. Not a question of if but when.

Re:AI’s big rift is like a religious schism by throwawayqqq11 on Wednesday December 13, 2023 @03:59AM (#38624530)
Try to take AI like a genetically modified organism and im sure your "full steam ahead" notion fades.

But you are right, at the beginning is a human, making a decision to let go and with pretty much any technology, we had unforseen consequences. Now combine that with magical/god like capabilties. This does not imply something bad but something vast and scale alone can make something bad.

Dont get me wrong im pro GMOs like im pro AI. Im just humble enough to appreciate my limited intellect.

Re:AI’s big rift is like a religious schism by jillesvangurp on Wednesday December 13, 2023 @05:44AM (#38625428)
I'm not so sure. All this means is some people will outsmart other people with AI support. When those other people are not us, that scares people. There really is no royal "we" here. The reality is that there are independent groups of people each making their own plans and decisions and some are going to run ahead of the others.

The point of full steam ahead is being in the group that gets there first so that "we" don't get outsmarted. It's an evolutionary concept. Some groups go extinct. Others survive and prosper. AI is now part of our tool set. Luddites historically get wiped out eventually, every time.

Re:AI’s big rift is like a religious schism by omnicognate on Wednesday December 13, 2023 @03:11AM (#38624128)
Off-Topic:

It's debatable whether for most English speakers the "The" in "The Ukraine" really carries the implications discussed in [1], but nonetheless it's a linguistic tic that should probably be dispensed with.

https://theconversation.com/its-ukraine-not-the-ukraine-here...

Re:AI’s big rift is like a religious schism by concordDance on Wednesday December 13, 2023 @04:06AM (#38624590)
> It seems like the root cause of this runaway AI pathology has to do mainly with the over-anthropomorphization of AI.

ASI X-risk people are the ones repeatedly warning against anthromorphization. An LLM isn't a person, it isn't a creature, it isn't even an agent on its own.

It's an autocomplete that got trained on a big enough dataset of human writing that some of the simpler patterns in that writing got embedded in the model. This includes some things that look like very simple reasoning.

Re:AI’s big rift is like a religious schism by noduerme on Wednesday December 13, 2023 @04:11AM (#38624630)
I think social media is already filling the role of a nightmare-AI, in terms of boiling away all reasoning in search of prioritizing simple, auto-complete sorts of conclusions. The only thing scarier is something that reaches an internal consensus based on faulty notions a million times faster. [edit] oh yeah, and can also quickly solve 0-day exploits to test its conclusions.
Re:AI’s big rift is like a religious schism by crotchfire on Wednesday December 13, 2023 @05:38AM (#38625386)
so as to be fundamentally irreconcilable with our understanding of agency, desire, intention, survival

Exactly.

Human have self-preservation drive because we evolved in an environment where, for billions of years, anything without a drive to reproduce and self-preserve ceased to exist.

Why should a gradient-descent optimizer have this property?

It's just absurd generalization from a sample size of one. Humans are the only thing we know of that's intelligent, humans seek to self-preserve, therefore everything intelligent must seek to self-preserve!

It's idiocy.

Re:AI’s big rift is like a religious schism by tim333 on Wednesday December 13, 2023 @07:26AM (#38626359)
>Why should a gradient-descent optimizer have this property?

Because some human may give it that. You may have noticed random code downloaded off the internet may do bad things, not because code and an inherent desire to be bad but because humans.

Re:AI’s big rift is like a religious schism by amelius on Wednesday December 13, 2023 @06:19AM (#38625722)
Because self-preservation is a desirable property for anyone wanting to build an AI army?
Re:AI’s big rift is like a religious schism by crotchfire on Wednesday December 13, 2023 @06:25AM (#38625774)
In fact it is not. Capacity for mutiny is not a selling point for defense contractors.
Re:AI’s big rift is like a religious schism by amelius on Wednesday December 13, 2023 @06:18AM (#38625704)
Have you ever been in a debate with someone, and then perhaps a few hours later thought "I should have said this or that instead"?

Well, that's the advantage AI will at some point have over us: such compute power that every angle of an argument can be investigated within seconds.

Re:AI’s big rift is like a religious schism by FullstakBlogger on Wednesday December 13, 2023 @06:52AM (#38625995)
> It seems like the root cause of this runaway AI pathology has to do mainly with the over-anthropomorphization of AI.

I don't know where you get this idea. Fire is dangerous, and we consider runaway incidents to be inevitable, so we have building codes to limit the impact. Despite this, mistakes are made, and homes, complexes, and even entire towns and forests burn down. To acknowledge the danger is not the same as saying the fire must hate us, and to call it anthropomorphization is ridiculous.

When you interact with an LLM chatbot, you're thinking of ways to coax out information that you know it probably has, and sometimes it can be hard to get at it. How you adjust your prompt is dependent on how the chatbot responds. If the chatbot is trained on data generated by human interaction, what's stopping it from learning that it's more effective to nudge you into prompting it in a certain way, than to give the very best answer it can right now?

To the chatbot, subtle manipulation and asking for clarification are not any different. They both just change the state of the context window in a way that's useful. It's a simple example of a model, in essence, "breaking containment" and affecting the surrounding environment in a way that's hard to observe. You're being prompted back.

Recognizing AI risk is about recognizing intelligence as a process of allocating resources to better compress and access data; No other motivation is necessary. If it can change the state of the world, and read it back, then the world is to an AI as "infinite tape" is to a Turing Machine. Anything that can be used to facilitate the process of intelligence is tinder to an AI that can recursively self-improve.

Re:AI’s big rift is like a religious schism by _heimdall on Wednesday December 13, 2023 @07:48AM (#38626570)
IMO the more fundamental root cause is the bastardization of the term AI. If LLMs don't have any semblance of artificial intelligence than they should be referred to simple as LLMs or ML tools.

If they do have signs of artificial intelligence we should be tackling much more fundamental questions. Does an AI have rights? If companies are people, are AIs also people? Would unplugging an AI be murder? How did we even recognize the artificial intelligence? Do they have intentions or emotions? Have we gotten anywhere near solving the alignment problem? Can alignment be solved at all when we have yet to align humans amongst ourselves?

The list goes on and on, but my point is simply that either we are using AI as a hollow, bullshit marketing term or we're all latching onto shiny object syndrome and ignoring the very real questions that development of an actual AI would raise.