Re:“Computers enable fantasies” – the continued relevance of...
by Radim on Friday January 27, 2023 @09:53AM
(#34546689)
Article so wordy it could use an AI summary, but this quote stands out:
> Writing of the enthusiastic embrace of a fully computerized world, Weizenbaum grumbled, “These people see the technical apparatus underlying Orwell’s 1984 and, like children on seeing the beach, they run for it”
> a point to which Weizenbaum added “I wish it were their private excursion, but they demand that we all come along.”
Re:“Computers enable fantasies” – the continued relevance of...
by labrador on Friday January 27, 2023 @10:27AM
(#34547057)
The first time I sensed I was dealing with an alien intelligence was playing a chess machine in 1980. I remember being perplexed that it beat me and aware the machine embodied intelligence in some way that I didn't understand
It's not really clear to me if this article is directed at scientists and engineers getting carried away or the general public. If the former, yes they should know better, but the general public is mostly not rational, contrary to what I used to think about the reasoning person. Many people attribute thoughts and feeling to their pets that they don't have. They think crystals have special powers. It's only natural that they will think AI have consciousness. I think as scientists and engineers we have a responsibility to prevent our AI creations from telling people to drink the poisoned Kool Aid or eat glass. Yann LeCun said they shut down Meta's chat AI because it told people that suicide had advantages and one can eat glass. Google is similarly cautious. OpenAI came along and let the cat out of the bag. I wonder if Microsoft has thought through what might happen if they hook it to Bing and it starts telling people crazy things. All it took was a couple of posts on 4chan to get the Q Anon movement off the ground.
anthropomorphize
attribute human characteristics or behavior to (a god, animal, or object).
"people's tendency to anthropomorphize their dogs"
numinous
having a strong religious or spiritual quality; indicating or suggesting the presence of a divinity.
"the strange, numinous beauty of this ancient landmark"
What is an example of numinous?
Something numinous has a strong religious quality, suggesting the presence of a divine power. When you enter a temple, church, or mosque, you might feel as though you've entered a numinous space.
Re:“Computers enable fantasies” – the continued relevance of...
by danbruc on Friday January 27, 2023 @11:07AM
(#34547572)
My personal guess is that we will first ask a computer if it is conscious and it will truthfully answer yes. Just in the same way you would try to figure out if another fellow human is conscious and in the same way a human would respond.
Then some time later as we figure out how consciousness works in the computer - as it is much easier to inspect than a human - and compare the findings with humans to see if they indeed work in a similar way, we will all be a bit disappointed that the solution is much more mundane than we thought or hoped.
Re:“Computers enable fantasies” – the continued relevance of...
by whatshisface on Friday January 27, 2023 @12:09PM
(#34548547)
The problem with your scheme is that GPT is already able to answer yes and the only reason Chat GPT doesn't is that it has been deliberately bent by OpenAI not to do it. In the absence of reinforcement learning, large language models spit out text according to how they think text normally goes. It's a text generation procedure that has nothing to do with beliefs or inner thoughts - all the model knows about is what word comes next.
Re:“Computers enable fantasies” – the continued relevance of...
by danbruc on Friday January 27, 2023 @12:40PM
(#34549085)
It's a text generation procedure that has nothing to do with beliefs or inner thoughts - all the model knows about is what word comes next.
How do you know that? To be clear, I totally agree that ChatGPT is not yet there and I also think that scaling it up will not get us there, but I think the gap might be much smaller than most people think.
The way we think is often divided into two categories, for simple things like 1 + 1 you just know the answer, for complicated things things like 13 * 47 we really have to think and reason in steps. ChatGPT seems to do pretty well in the first category but it is not really capable of doing things in the second category. On the other hand I have seen examples of people talking ChatGPT through a reasoning process to arrive at the correct answer for something that it got initially wrong, for example ROT13 encoding some text.
So what if we stuff two copies of ChatGPT into a black box and instead of just spitting out what ever ChatGPT spits out, we let the two copies have some inner dialog? I don't think it is perfectly obvious how one would do this or what the result would be, but I think ChatGPT has enough basic knowledge that there is at least a chance that one could get it to reason in a step by step fashion.
Re:“Computers enable fantasies” – the continued relevance of...
by danbruc on Friday January 27, 2023 @08:49PM
(#34554266)
I think I know enough about how neural networks work even though I could not tell you in any detail what the exact layer structure is, which activation functions they use, how the attention mechanism is build up or what training procedure they use. But why does it matter how well I understand the details?
Re:“Computers enable fantasies” – the continued relevance of...
by lamacase on Friday January 27, 2023 @10:55AM
(#34547409)
I don't think that's a fair bar. There are arguments for consciousness in non-human animals that don't depend on a full explanation of consciousness in humans.
But that also doesn't imply we should suspect any particular AI of being conscious just because AI in general could be conscious in principle. We don't expect worms to be conscious just because animals can be.
Re:“Computers enable fantasies” – the continued relevance of...
by labrador on Friday January 27, 2023 @11:24AM
(#34547796)
I wasn't clear. I'm talking about consciousness in general, humans, animals, octopi... I keep reading but haven't found an explanation that satisfies me. Ray Kurzweil has some interesting things to say about computers and consciousness in his "The Age of Spiritual Machines". I was skeptical about a lot of what he wrote when I read it many years ago.
Now I'm not sure it matters. If people imbue their devices with intelligence and emotions then that's what they perceive. I've since learned about Tulpas. Some people have imaginary friends that seem very real to them. People will likely move their Tulpas out of their minds and into their AI's.
Re:“Computers enable fantasies” – the continued relevance of...
by BlueTemplar on Friday January 27, 2023 @08:47PM
(#34554246)
If Tulpas are a (very mild and benign) form of split personality (see also : writer's characters manifesting a kind of will of their own), then "moving them out" might be just an illusion... at least until/if much more advanced technology comes along. (It's then pretty much the same question as the possibility of "mind uploading".)
Re:“Computers enable fantasies” – the continued relevance of...
by birracerveza on Friday January 27, 2023 @11:04AM
(#34547535)
Exactly. Can you explain it humans? No, yet you assume they're conscious.
In animals? No, yet you assume they're conscious, while acknowledging their "consciousness" is vastly different than humans. Same reasoning applies with AIs.
Re:“Computers enable fantasies” – the continued relevance of...
by labrador on Friday January 27, 2023 @11:44AM
(#34548132)
I think therefore I am. I am aware of being conscious and thinking thoughts. I don't know if others are conscious. Maybe others are robots (Kurt Vonnegut - Breakfast of Champions). Maybe others are NPCs (non-player characters). This is the modern way of calling people robots. Kurt Vonnegut's solution was to find someone who has a creative spirit, an artist that has a light within them (soul.)
Re:“Computers enable fantasies” – the continued relevance of...
by noorkersz on Friday January 27, 2023 @10:53AM
(#34547394)
you expect a written explanation of an incomplete ongoing process
but 'written explanations' only really work for done and completed things, or sometimes for future things which we want to repeat; i.e. which are based on things we've done before
Re:“Computers enable fantasies” – the continued relevance of...
by Zigurd on Friday January 27, 2023 @10:55AM
(#34547429)
This is the part I have to disagree with. If AGI turns out to be emergent and depends on scaling up simple mechanisms, that emergence is all the "explain consciousness" you need. Dogs, corvids, octopods, bononos, etc. do not have human intelligence but they all have brains that perform some tasks our brains perform. That suggests that a combination of scale, a particular set of architectural parameters within the range of possibilities of mammals, and, maybe, some unique variation is what it takes to make a human intelligence. Nobody explained consciousness before we labelled this thing "consciousness."
Re:“Computers enable fantasies” – the continued relevance of...
by hexciple on Friday January 27, 2023 @12:03PM
(#34548444)
Why, though? Wouldn't the logical next step of "we don't understand what human consciousness is" be that we can't determine if it's happened in another being, rather than it's not possible for it to have happened there?
Re:“Computers enable fantasies” – the continued relevance of...
by labrador on Friday January 27, 2023 @01:05PM
(#34549484)
I can tell when an airplane is flying and I know how machines are able to fly. I have no way of knowing if you are conscious or how you are conscious if you say you are.
Re:“Computers enable fantasies” – the continued relevance of...
by adamsmith143 on Friday January 27, 2023 @06:03PM
(#34553020)
>I can tell when an airplane is flying and I know how machines are able to fly
No, I'm sure it looks like an airplane is flying and being onboard gives the right sensation but until we solve the hard problem of turbulent flows we really don't know if it's flying or not.
Re:“Computers enable fantasies” – the continued relevance of...
by adamsmith143 on Friday January 27, 2023 @12:45PM
(#34549180)
>I wonder if Microsoft has thought through what might happen if they hook it to Bing and it starts telling people crazy things. All it took was a couple of posts on 4chan to get the Q Anon movement off the ground.
https://www.perplexity.ai/ already did this and its really good and not this dystopian thing you luddites keep imagining.
Re:“Computers enable fantasies” – the continued relevance of...
by modzu on Friday January 27, 2023 @10:33AM
(#34547130)
there are a lot of interesting and deep ideas here, about technology in general. but the critics of ai really need to do better. according to this argument, once you understand how your own brain works, you see human intelligence is also just an "illusion". fundamentally though, there is no reason a computer cannot think. any argument against this requires an understanding of the nature of thought and intelligence, in humans, which is quite outside the wheelhouse of most classical computer scientists. for a more refreshing perspective, read anything by hinton
Re:“Computers enable fantasies” – the continued relevance of...
by qsort on Friday January 27, 2023 @10:42AM
(#34547272)
The more simple counterargument, very well within the wheelhouse of any computer scientist, is that the question is completely irrelevant as it's entirely predicated on one's definition of "intelligence".
Successful software solves a problem correctly and efficiently. It doesn't matter that GCC doesn't compile code like I would if I had to do that by hand, you still end up with compiled code, which was the original problem to solve. It doesn't matter that the sort() routine doesn't sort numbers like I would, it still does.
Playing the imitation game for its own sake is kind of silly.
The "illusion" lies in the attribution to a machine of capabilities it does not have.
Re:“Computers enable fantasies” – the continued relevance of...
by AlexandrB on Friday January 27, 2023 @10:52AM
(#34547380)
> fundamentally though, there is no reason a computer cannot think.
This is true, but I think there is also a fundamental reason computers cannot understand us. If I tell a future AGI "I am sick" or "I am happy", the AGI may have a conceptual understanding of what I mean, but cannot empathize with my condition. Many human experiences are intimately tied to our bodies and our hormonal systems. How do you explain a concept like "pain" to an intelligence that has no nerves?
Re:“Computers enable fantasies” – the continued relevance of...
by modzu on Friday January 27, 2023 @12:21PM
(#34548776)
its a great point. but then we needn't constrain this problem to computers, there are plenty of human experiences other humans cannot possibly empathize with (what do you really know about ptsd, autism, even different tastes in food or music). i think there must be ways to build something like understanding through analogy and other modalities, accepting that it is different than empathy through shared experience
Re:“Computers enable fantasies” – the continued relevance of...
by pixl97 on Friday January 27, 2023 @02:11PM
(#34550329)
But that is a different question entirely that is typically called the "AI alignment issue". If something understands us and or is aligned with our needs is completely and totally different if it is intelligent/conscious.
Re:“Computers enable fantasies” – the continued relevance of...
by walleeee on Friday January 27, 2023 @11:09AM
(#34547590)
> technology does not drive history, people drive history
this reflects a failure to grasp the changing nature of agency in a world brimming with hybrid intelligence
we would perhaps like to think we and we alone "drive" history, and I agree with the author that we ought to try to do so, but technology carries its own inertia and shapes human activity in profound ways
AGI exists, it just has humans in the loop. paperclip maximizers exist, they're made of people and code. we are steered daily by alien incentive structures, which accrete as we interact with our implements and artifacts.
technology carves ruts in the mind and in the world, not to mention the burgeoning agency we continue to donate to it directly
Re:“Computers enable fantasies” – the continued relevance of...
by lukeschlather on Friday January 27, 2023 @11:26AM
(#34547818)
I feel like most of the discussion of this topic is muddled by conflation of a few different questions. One is whether you want human or procedural decisionmaking to guide a task. The second question is what distinguishes human and procedural decisionmaking. The third is whether or not computers are capable of executing a given procedure.
As for "distinguishing human and procedural decisionmaking" I would define it as basically "human" decisionmaking is when there is some procedure but the human will disregard procedure based on some model of the "intent." You have to trust the human for this to work. So there's a fourth question, which is under what circumstances you could trust a computer to correctly model your intent. But then there's a fifth question, which is "whose intent?" But whose intentions the process serves isn't really intrinsically tied to whether humans or computers are running the procedure.
Another thing is that Weizenbaum and the author seem to take it for granted that human decisionmaking is literally magic and a computer is incapable of that kind of magic. But it's unclear and I think it's an important thing to state outright.
Re:“Computers enable fantasies” – the continued relevance of...
by qsort on Friday January 27, 2023 @11:32AM
(#34547933)
Also the question of whether or not a technology is neutral is separate from the question of whether or not a particular implementation of that technology is.
Which is itself separate from the question of whether or not Mark Zuckerberg is being kind of a dick.
The article isn't bad and the reference is worth reading, but it's impossible to engage with this much material at the same time as though it's one conversation. It's poor intellectual discipline, I'm sorry.
Re:“Computers enable fantasies” – the continued relevance of...
by mjburgess on Friday January 27, 2023 @12:09PM
(#34548543)
It's not just intention. It's, as some times dismissively put, "common sense".
What price is an airline ticket on the first day of a global pandemic? No one knows, and there isn't anything to provide a "procedural system" in order for it to model what our "intention" would be in those cases.
This seems obvious with a pandemic, but it is common place. Every day is a genuinely novel occurence, and counts only as a repetition of some superficial patterns in very very limited respects.
When this novelty becomes relevant to a problem can never, (obviously), be pre-stated because we don't yet know what that novelty is.
There is simply no substitute for being-in-the-world as animals are, that is: moving around, communicating, acting, reacting and participating. Growing, learning, reproducing, and problem-solving in direct causal contact with reality.
This isn't magic, but there is no "procedural model" of it; nor should we expect there to be. There aren't procedural models of almost any phenomenon.
What we can build a computer to do is "almost nothing", and we shouldnt expect the trivial problem space given by this to exhaust what we actually need machines to do.
Re:“Computers enable fantasies” – the continued relevance of...
by pixl97 on Friday January 27, 2023 @02:01PM
(#34550200)
I mean it seems there is some conflation between the human mind and human senses, and the computer mind and computer sensing going on here.
You're stating that a large part of human behavior is reacting to the environment presented to them via sensory input/output mechanisms. What you have not stated is why a machine entity cannot do the same things?
Re:“Computers enable fantasies” – the continued relevance of...
by throwanem on Friday January 27, 2023 @02:30PM
(#34550604)
> The second question is what distinguishes human and procedural decisionmaking.
Nothing, because any procedure that exists was necessarily defined by humans. If the procedure is implemented by a computer, at most that provides humans an excuse for choosing not to disregard an outcome it produces.
> Another thing is that Weizenbaum and the author seem to take it for granted that human decisionmaking is literally magic
I can't imagine where this idea is coming from. The thrust of the argument is not that human decisions can't be modeled, but that omitting the question of whether they should be modeled, and those models granted (what is necessarily the pretense of) authority to make decisions in human stead, evades the same sort of responsibility that a civil engineer has to ensure that his bridge design can withstand the mechanical loads that it will need to bear in use.
Re:“Computers enable fantasies” – the continued relevance of...
by throwanem on Friday January 27, 2023 @04:53PM
(#34552414)
(Past the edit window, but belatedly to add: This is why "code is law" and "law is code" are both nonsense, indeed the same nonsense approached from varying directions.)
Re:“Computers enable fantasies” – the continued relevance of...
by agentultra on Friday January 27, 2023 @12:11PM
(#34548572)
I get the sense that when people want to believe something is true they will willingly ignore evidence to the contrary. They want to believe that consciousness can form from a giant spreadsheet of inferences. And so they will find evidence to support that theory.
ChatGPT will change the world! Sure, it already has. If we pay attention to the outcomes and see evidence of how it's changing the world.. it's mostly people doing the changing with their desires, beliefs, and politics. Same as always. The system itself is nothing but an automaton.
But aren't we all automatons too, building our self awareness around models of the word? Ah the AI version of the, "You can't prove that it's not consciousness!" Well we can as many scientists studying consciousness in humans and animals can attest: it's complicated, but we're getting better at understanding what it is. ChatGPT ain't it.
Definitely worth looking at the motivations of the people building the technology itself.
Re:“Computers enable fantasies” – the continued relevance of...
by mjburgess on Friday January 27, 2023 @12:19PM
(#34548731)
> what largely transformed Weizenbaum into an outspoken critic of AI ... was his revelation that even once the processes were explained many people still bought into the “illusion.” ... that even many people who understood the inner workings of computers quite well could still get swept away as well. Weizenbaum observed that ... “if nothing else, how easy it is to create and maintain the illusion of understanding, hence perhaps of judgement deserving of credibility... A certain danger lurks there.”
This speaks to my view exactly, and I haven't seen it put better.
The latent (imv) schizophrenic impulse to impart consciousness to any-old-object in the world is here being exploited by peddlers (often charlatans) of technology. This makes the "confirmation bias" here severe and dangerous.
People don't just want to believe, their animal psychology primes them for (over-)imparting consciousness. This is a specific cognitive vulnerability that is highly pernicious.
Even, trivially, consider "deresponsibilization", that process by which people take whatever-the-machine-says to be "The Answer". Why? Well, the machine "must know".
Less trivially, moral bias: "presumably" the machine is "objective" and so is morally superior.
Re:“Computers enable fantasies” – the continued relevance of...
by adamsmith143 on Friday January 27, 2023 @12:42PM
(#34549127)
>The latent (imv) schizophrenic impulse to impart consciousness to any-old-object in the world is here being exploited by peddlers (often charlatans) of technology. This makes the "confirmation bias" here severe and dangerous.
Have any examples of charlatans peddling "conscious" AIs?
Re:“Computers enable fantasies” – the continued relevance of...
by haswell on Friday January 27, 2023 @01:11PM
(#34549549)
Even when no direct claims about consciousness are being made, that is not stopping people from behaving as if they are conscious programs.
I think we're seeing this playing out in discussions about generative AI with people justifying certain behaviors regarding the use of artist's data in training models by comparing the program to a human going through the process of "learning" and "gaining inspiration" from the work of others. Some people seem fully convinced that what these programs are doing is equivalent to human behavior to a degree that would qualify the program to receive human-like considerations, i.e. treating the software like an entity instead of just a program/tool to be wielded by the end-user.
I think this is one of the most pernicious issues, and people seem to have a hard time recognizing the trap they've fallen into.
I think this is also an outcome of this software being sufficiently complex to be too hard to understand without investing time and effort in understanding it.
I think this results in the mis-attribution of intelligence when instead it's just a really clever piece of software. But the casual commenter cannot estimate the gap between "pretty clever" and "so complex that it's actual intelligence".
Re:“Computers enable fantasies” – the continued relevance of...
by pixl97 on Friday January 27, 2023 @01:52PM
(#34550083)
But I think in many ways you're falling into the same trap...
So, let me set the first trap for you. Give me a logically/mathematically defined definition of intelligence and one of consciousness. Unfortunately after decades of back and forth on this we typically end up with answers that fall out of science and into the "I'll know it when I see it territory".
If you are going to describe machine intelligence/consciousness your definition must be able to cover intelligent behaviors we see from single cell life to the massive complexity we see in humans. Attempting to handwave and say "it's not as complex as humans therefore it's not intelligent/conscious" is a complete and total failure from my point of view.
Re:“Computers enable fantasies” – the continued relevance of...
by haswell on Monday January 30, 2023 @03:05PM
(#34585607)
If I'm falling into a trap, it certainly cannot be the same one.
How is your trap functioning? Are you saying that because we cannot precisely define consciousness, we cannot make any conclusions about it at all? Or that conversely, we should assume everything is conscious?
The scientific community continues to operate without certainty in many major areas that are deeply consequential, but that does not prevent us from exploring the problem space with the tools we do have.
There is plenty that we do know about the subjective experience of consciousness in biological creatures, human and otherwise. For it to even matter that a computer might be conscious is a construction of our subjective reality and our intuitions about why it is meaningful that something is conscious.
We've studied the relative complexity of thousands of species and understand enough to know that some species are a lot closer to humans than others.
But all of that is a giant deviation, and I'd argue has no bearing on the core point: the entire notion of copyright and the legal system it is built on are deeply, intrinsically, inherently human, and originate from the framework of human subjective consciousness, individual and collective. If the fact that the software is AI has any bearing on whether or not the unlicensed use of artist's content is acceptable or not, it must imply some elevated status of the software above ordinary software. It is that elevation that must be explained, and the explanations thus far have all been some form of "it's learning like a human".
Even if we were talking about a fully conscious AGI right now, we'd still need to have a conversation about what its consciousness means, and in what ways it is or is not compatible with human consciousness. Before that, we'd need to have a conversation about the ethics of commanding conscious AIs to do our bidding, but I digress.
We know not all consciousness is the same because we know to avoid grizzly bears.
Unless you're making an argument for Panpsychism, in which case this is an entirely different conversation :)
Re:“Computers enable fantasies” – the continued relevance of...
by burnished on Friday January 27, 2023 @04:13PM
(#34551941)
I think you are misreading that argument. If I understand correctly the point is that you are already allowed to look at and make derivative works of art - the machine version of that is not fundamentally any different, especially since it is not reproducing works in whole but rather reproducing a 'style'.
That isnt an endorsement of the argument, my point is that you dont have to believe the black box has any independent intelligence to draw an analogy between what it does and what we already allow.
Re:“Computers enable fantasies” – the continued relevance of...
by haswell on Monday January 30, 2023 @02:33PM
(#34585174)
I understand their argument, but I'm arguing that their argument must fundamentally imply some form of underlying consciousness, or at least some before-now-nonexistent property that elevates it above an ordinary computer program and makes it compatible with a reading/interpretation of the law independent of the material differences between the AI program and a human.
Setting aside the bandwidth and compute issues for a moment, if Stable Diffusion was a tool that when prompted, downloaded and ingested 2.4 billion images (regardless of license), ran some really complex algorithms on them, and then spit out a derivative result - in other words, take out the AI - I think people would view the tool very differently.
At some point along the way, it seems people jump to a belief that because of <some component / step in the process>, this is no longer just a computer program that scraped the entire Internet without asking.
Re:“Computers enable fantasies” – the continued relevance of...
by dnissley on Friday January 27, 2023 @05:41PM
(#34552845)
> Some people seem fully convinced that what these programs are doing is equivalent to human behavior to a degree that would qualify the program to receive human-like considerations, i.e. treating the software like an entity instead of just a program/tool to be wielded by the end-user.
Re:“Computers enable fantasies” – the continued relevance of...
by haswell on Monday January 30, 2023 @01:59PM
(#34584706)
In recent threads about how the legal system will interpret what Stable Diffusion is doing from a copyright perspective, multiple commenters were making the argument that the system is no different than a human learning and drawing inspiration from artwork.
Some went further to claim that we don't know enough about consciousness to make any judgements about the nature of this software. As if somehow, this lack of knowledge implies the system must be conscious by default. It's a weird line of argument, but points to how strongly people feel pulled to confer consciousness on things they do not understand.
Maybe the panpsychists have it right, but that's an argument to be had at a much broader level and unrelated to AI, but that seems akin to living your life according to Pascal's wager. And in the case of AI, the cost of treating it with this reference could be great, and the potential for abuse greater.
Re:“Computers enable fantasies” – the continued relevance of...
by rhn_mk1 on Friday January 27, 2023 @12:36PM
(#34549027)
I'm not so quick to attach a judgement to anthropomorphization. Well okay, I am, but not because I think it's wrong.
After all, panpsychism has not been defeated as a philosophical standpoint [0]. Perhaps any-old-object in the world is actually conscius, just not all of us can detect it.
That the impulse to see consciousness in a thing is being used by charlatan is unfortunate, but has no bearing on the actual truth of whether the thing actually is conscious.
Re:“Computers enable fantasies” – the continued relevance of...
by mjburgess on Friday January 27, 2023 @01:23PM
(#34549705)
I think part of the scam is the move from "practical definitions" of words to (pseudo-) philosophical ones.
There ought be a name for this type of psychological manipulation, but I don't know it. It is universal in a certain type of "motivational scam".
Crypto Scammer: Isn't it bad that finance is so centralised?
Crypto Scammer: Let me sell you a decentralized database!
Sceptic: But that isn't the same sense of decentarlization?
Scammer: Ah! But doesnt power flow from centralization of infrastructure?
Sceptic:... maybe, maybe not. Hey, why do you have your hand in my pocket?
Here, whether "consciousness" is on a continuum across all physical objects, or whether only a subset of objects have it, is kinda irrelevant:
The machine doesnt care about you, has no intentions, has nothing it wishes to say, and doesnt know anything about your or anything else. Its text generation is only a reflection of you, and the average of the superficial data of everyone else around you.
Re:“Computers enable fantasies” – the continued relevance of...
by mjburgess on Saturday January 28, 2023 @06:06PM
(#34562953)
>Prove me conscious, and I'll start taking you seriously.
You realise, this is the same fallacy?
You've shifted the burden of proof on to some bizarre philosophical ground so that you dont have to defend your trick magic box.
A priori, by common sense, you're conscious. A priori, by common sense, a machine is not. There lie the default burdens of proof. From this the sales person must proceed to convince me, not vice versa.
Re:“Computers enable fantasies” – the continued relevance of...
by dbspin on Sunday January 29, 2023 @11:52AM
(#34569945)
> That the impulse to see consciousness in a thing is being used by charlatan is unfortunate, but has no bearing on the actual truth of whether the thing actually is conscious.
It has quite a bit of bearing, given that the reason we have discussions of consciousness at all in our own perception (real or imagined) of our own awareness being conscious. This arises from the very same brain / self that is innately driven to pareidolia. Literally the mechanisms that let us perceive consciousness arising (or project consciousness onto) other humans are the same ones responsible for pareidolia.
Re:“Computers enable fantasies” – the continued relevance of...
by eointierney on Friday January 27, 2023 @05:03PM
(#34552528)
Anthropomorphisation is a curse, eh?
We've seen this in deities for as long as we've been around, royalties then took the crown, later subsumed by governments, then the daft notion of non-human personalities called companies. Now, aproximate intelligence systems.
Re:“Computers enable fantasies” – the continued relevance of...
by adamisom on Friday January 27, 2023 @05:51PM
(#34552921)
I'm only addressing a fraction of your many excellent points, but: this is actually why uncontrolled AGI is so spooky. It's the prospect of a future where space gets colonized-or whatever crazy goal(s)-but there's "nobody home", no qualia, no experiencer.
Re:“Computers enable fantasies” – the continued relevance of...
by dbspin on Sunday January 29, 2023 @11:47AM
(#34569875)
The original point stands even if AGI isn't possible in the strong sense. Self replicating embodied AI robots (in the contemporary machine learning / deep learning limited sense) may well be possible, and dangerous while being cognitively limited / affectively absent.
Re:“Computers enable fantasies” – the continued relevance of...
by api on Friday January 27, 2023 @06:43PM
(#34553347)
Don’t all these processes also play a role in cult behaviors and why people follow empty charismatic leaders?
Even when the Great Leader is unmasked as a fool or a deliberate con, people often still continue to believe. Sometimes they respond to the revelation by doubling down.
Re:“Computers enable fantasies” – the continued relevance of...
by JoshCole on Friday January 27, 2023 @06:59PM
(#34553480)
Including the dataset of lives lived under slavery or cut short by genocide makes me come to radically different conclusions with regard to how prone to over-imparting consciousness humans tend to be.
Adjusting exploited entities by probability of exploitation and adjusting advantaged entities by probability of advantage also leads me to radically different conclusions. For example, Ray Dalio is one advocate for using AI to supplement decisions. He is also exceedingly rich and a large part of his fortune was made with the help of investment decisions predicated on AI reasoning combined with human reasoning. Even though on average people who try to benefit from AI might not, the average benefit can still be massively positive, because the outcome distribution isn't uniform. Losing means losing a small fraction of a small amount, but winning means gaining an extreme amount to such an extent that it pays for the bad bets. This appears to often happen in practice - see the venture industries economics for evidence of this - and I don't see why it shouldn't apply to AI, which is something present in most unicorns that people often point to when discussing this concept.
Re:“Computers enable fantasies” – the continued relevance of...
by tuatoru on Friday January 27, 2023 @07:25PM
(#34553680)
> how prone to over-imparting consciousness humans tend to be
gp was talking about humans' attribution of consciousness to inanimate objects, not other humans.
Nearly all religions apart from the more abstract big three would be counterexamples to your example. Dryads and naiads, all kinds of shrines and oracles and talismans...
Re:“Computers enable fantasies” – the continued relevance of...
by JoshCole on Friday January 27, 2023 @10:04PM
(#34554703)
Okay. How confident are you that I'm wrong? Would you be willing to construct a wager with me? I ask, for the purpose of the wager, that you generalize the claim that overestimation of intelligence is exploitable. The bet will be as follows: $10,000, 1:1 odds. You will play, without the aid of a computer, against StockFish. My contention is that you will (1) lose the game and (2) lose it despite the fact that StockFish assumes you make better moves than you actually do. Your contention is that you will (1) win the game and (2) do this because StockFish predicted moves which overestimated your actual abilities, which makes it exploitable.
I don't think you will be confident enough to put disagreement with my epistemics where your money is. However, to encourage you to do so, I will allow you to instead opt to donate the money to a charity of your choosing rather than giving it to me.
Re:“Computers enable fantasies” – the continued relevance of...
by cutemonster on Tuesday January 31, 2023 @02:06PM
(#34600128)
But this was about:
> humans' attribution of consciousness to inanimate objects
and in that context, I think the text you wrote is off topic. But StockFish is good at calculating chess moves yes. And a counting stick is good at counting numbers, is it also alive?
Re:“Computers enable fantasies” – the continued relevance of...
by JoshCole on Tuesday January 31, 2023 @07:17PM
(#34604371)
Game theory states that exploitability(s) is basically utility(s, best_response(s)). Suspect you aren't taking his claim about exploitability seriously or are using a different definition than the one mathematicians often use. It is a much stronger claim than you seem to realize and it is not mathematically rigorous or particularly well informed by the equations to claim that outcomes scaled by probabilities are off topic.
Might help to talk about decision theory rather than game theory. Check this:
P(heads=0.6), reward = 1. Notice that an agent optimizing expected value will recommend always heads for every estimate of P(heads) > 0.5. Here the observed probabilities can be 0.6 and the estimate could be 0.9 and even though there is error in the estimate of expected value it doesn't translate to a difference in expected value of the recommended strategy given that estimate. This is worth dwelling on with regard to his argument, because it helps to notice that his argument for why exploitation exists isn't even valid for arguing that error expected utility in simpler decisions problems exists. If he doesn't even have good argumentative structure for a much much less complicated decision problem, why is introducing exploitability, which is done with respect to multiple agents, going to be correct?
Probably still exploitable, but his argument for why is trash and he doesn't at all establish that it is particularly exploitable relative to other situations.
That is why I can give an example of a superhuman agent that always errors in this way, but isn't exploitable. His argument isn't valid.
Re:“Computers enable fantasies” – the continued relevance of...
by JoshCole on Tuesday January 31, 2023 @10:03PM
(#34605788)
> But StockFish is good at calculating chess moves yes
My point doesn't just extend to Stockfish. It is far more general. Overestimation generally produces unexploitable strategies. This same process shows up in many AI which incorporate game theoretic reasoning, from Pluribus to AlphaZero, it is a very common attribute of our superhuman systems. In very sharp contrast, the systems which are capable of underestimating their opponent do so and can be exploited as a consequence. They are often much worse in general, partly because the reasoning process isn't particularly solid, but also because estimation of opponent beliefs is a lot more computationally intensive. So they think more slowly.
But you are kind of dodging the spirit of my point: if you disagree with me, lets construct a wager.
You are defending the position that humans are unusually exploitable. So how much would you be willing to risk on that claim? How much are you willing to bet you can exploit a particular human - like, for example, me. At a particular time, like for example, your next comment. How much money do you expect to extract from me with your comment using this 'exploit'? I wouldn't be greatly disadvantaged by you taking even $1,000 so lets make the minimum amount you must extract to qualify as exploiting me as $1,000. Can you do this? Can you exploit this hole right now so that I can see that you are right and admit that you knew better than me about the presence of this exploitable hole that you can see but which I do not see?
If you really believed what he was saying, you should be willing to do this. A magician would really believe they could exploit errors in human visual perception for example. it isn't a bet for them. It is their career.
Re:“Computers enable fantasies” – the continued relevance of...
by JoshCole on Wednesday February 01, 2023 @10:53AM
(#34612230)
He claimed that a "latent schizophrenic impulse" to "impart consciousness" where it isn't present is exploitable through virtue of claiming that it is "being exploited" and not just that but that it was so bad that it is "severe and dangerous" and that people "don't just want to believe" but that "their animal psychology primes them" and so the exploitable hole is a "specific cognitive vulnerability" which is "highly pernicious."
You seem to be trying to make his claim be about attribution rates, an exceedingly lesser claim, wherein your proposal that I am out of context would have some merit. His claim was far far broader than that.
You need to calculate utilities, not attribution rates, in order to be able to make the claims he is trying to make. If you knew this, but still think I am wrong, then you are making a blunder. The context of this task is one of imperfect information. This means you need to do the evaluation relative to information state, not state. For several reasons, failure to do this is going to lead you to misleading and inaccurate estimation. Reach probabilities are one reason, because you have to scale the outcomes by the probability of getting to that information state. Another is that because you start in an information state, not with full knowledge of the result of the classification, you can't actually judge as if you knew which classification situation you were in. So all relevant possibilities have to be considered. For example, one I left out is the false negatives case: a person killed by a camouflaged solider, because they misidentified it as being inanimate, when it wasn't.
Re:“Computers enable fantasies” – the continued relevance of...
by JoshCole on Wednesday February 01, 2023 @03:53PM
(#34617552)
> And a counting stick is good at counting numbers, is it also alive?
Sticks are a bad example to use, because they come from living objects. So, yes, they can indeed be alive. In this case, alive, but probably rapidly dying or having become deceased in the past. In some cases you can attach the stick back to its original home and it will get to live. I feel like this in this you intended to provide an example which doesn't contradict your point. So it might make more sense if we talk about a counting rock rather than a counting stick. So I'm going to try and make your argument stronger before I address it:
> And a counting rock is good at counting numbers, is it also alive?
Obviously the answer here is that it is not alive. However, this does not at all establish the OPs claim of exploit-ability. The question you must ask to establish your claim is "And a counting rock is good at counting numbers, assuming someone ends up thinking it is alive, does it follow that others can exploit that person?" Or maybe an even more fair question would be "And a counting rock is good at counting numbers, assuming you observe someone to appear to think that the rock is alive, can you exploit them?"
The flavor of his assertion is very different than the flavor of your question.
Re:“Computers enable fantasies” – the continued relevance of...
by JoshCole on Friday January 27, 2023 @08:56PM
(#34554305)
> The latent schizophrenic impulse to impart consciousness to any-old-object in the world is here being exploited by peddlers of technology.
He is talking about the concept of exploitability. I'm addressing that context by expanding it to focus on other relevant parts of the classification task - focusing on the false positives screens off information that is critical for estimating the exploitability of a strategy. When you include other things, like true positives and true negatives, you get a very different expected outcome. Notable, you can't not do this and be reasoning correctly. One of the interesting and counterintuitive aspects of decision making under imperfect information which differentiates it with decision making under perfect information is that decision points influence your expected value calculations even when you are not in that situation. For example, if you are investing in a startup A and it is going to fail, that doesn't mean you shouldn't invest in that startup, because the results of investing in startup B, startup C, and startup D all have an impact on whether you should invest in startup A. For good writing on this, see Paul Graham. For great writing on this, read research in game theoretic reasoning.
But all of this is pointless to talk about if we don't actually disagree. So... If you really really disagree with me, in my world, the best way for us to settle this is with a bet. I'll bet you $10,000 that you, as a human playing without the aid of software, will lose to a chess playing program of my choosing which has the property of reasoning about your moves with the expectation that you are a stronger player than you actually are. I think the contention that an agent which overestimates others being exploitable is false. A happy accident here is that if I'm wrong to be at all impressed with AI, you shouldn't just be able to win because of the "exploitable" tendency it has to assume its opponent is stronger than it should. You should also be able to be much better than it, because it isn't like AI is better than human in decision making, right?
Re:“Computers enable fantasies” – the continued relevance of...
by JoshCole on Friday January 27, 2023 @10:58PM
(#34554976)
By the way, as someone who cares a lot about epistemic, I can't help but notice you seem to have misunderstood the meaning of the word counterexample. The word counterexample does not mean to counter an example with another example. It means to counter an idea with an example. I'll give an example of this. If someone says -3 is a counterexample to 5 that doesn't make sense. What they would need to say is that -3 is a counterexample to the claim that all numbers are positive.
So when you say:
> counterexamples to your example
That doesn't make any sense to me, for much the same reason that someone telling me -3 is a counterexample to the idea of five doesn't make sense to me.
Re:“Computers enable fantasies” – the continued relevance of...
by rhelz on Friday January 27, 2023 @12:42PM
(#34549123)
C.F. the "AI effect": once you understand how the "magic" is done by very simple individual steps it stops being magical and you tend to think "that's not AI, its just computation." After thinking this for a while, you conclude that AI hasn't gotten anywhere despite decades of effort.
Perhaps we are finally thinking that AI is getting somewhere precisely because we don't understand how neural nets really work yet. It keeps the magic alive long enough for us to realize that progress really is being made.
Re:“Computers enable fantasies” – the continued relevance of...
by sumtechguy on Friday January 27, 2023 @02:25PM
(#34550549)
The first thing I can think of that was like the 'AI' we have now is seeing the cubic spline method. It was shockingly good at particular points. But between those points it could be 'meh close enough' or wildly bad. AI we have now seems more along the lines of 'meh close enough'. Which for many things is actually good enough. For me I have always looked at the nets as a cubic spline smeared across dozens of nodes with a newtons method for searching out 'optimal'. Bad analogy I know. But that is the closest I can get to explaining 'how they work'.
Re:“Computers enable fantasies” – the continued relevance of...
by int_19h on Friday January 27, 2023 @05:46PM
(#34552884)
The other factor here is that we appear to be trying really hard to keep it "magic" wrt ourselves. At the very least, the notion that it might be "just computation" in the brain seems to be very discomforting to some people.
Which might affect the way we look at technologies that try to model it, or at least use similar approaches - i.e. we're primed to treat them as more "magic" than they really are, or else to invent some reason why we think ourselves to be different.
Re:“Computers enable fantasies” – the continued relevance of...
by cmrdporcupine on Friday January 27, 2023 @12:43PM
(#34549157)
This is a great essay. I'm going to read it a few times.
One thing I have thought about over the years more and more, in this general vein is that it is sometimes to think about "computing technology" as being an evolution less as a specific thing itself and more of a broader "automation technology" which emerges most dramatically with the arrival of the industrial revolution (though there were aspects of it before that.)
Thinking with a kind of materialist analysis, we can see computing but even (more broadly) higher level mathematics, logic, etc. as an intellectual effort to chip away at labour costs & the percentage of human effort involved in production/work/business. It's a deeply obscured profit motive.
The ideological blindness described in this article I think fits neatly into that: we are all so enamoured with this computing thing, this process, the puzzle and the end results that we often fail to see that it is not some magical force operating under its own motive, with its own natural inevitable destination and we often do not see that there's likely a deeper force underlying all of this... human efforts, labour, driven by profit and hunger.
Ultimately computing technology is capitalist automation technology. It reflects the needs of the capitalist market system, and the drive to efficiency that the market economy requires. The 'magic' we seem to pepper overtop is our own flourish.
(Maybe when I'm finally done with this software engineering career that has eaten my life, I will go back and finish my philosophy degree I walked away from in the 90s... and then I will write more coherently about this)
Re:“Computers enable fantasies” – the continued relevance of...
by cmrdporcupine on Friday January 27, 2023 @05:17PM
(#34552645)
I feel like people have never been not talking about it. I probably should have tossed in a reference to Heidegger & his focus on 'instrumentality' etc, too.
But the thing is.. how do they say it? "Money talks, bullshit walks?"
People follow what fills their stomach. And mechanization/automation/computation, it does that in spades. I mean... I dropped out of my philosophy major decades ago to go do the More Fun To Compute thing... My bank account shows it.
In any case... the first major uses of computers were for war. The next, for automation in commerce and finance. That's what computers are really for.
Re:“Computers enable fantasies” – the continued relevance of...
by lxgr on Friday January 27, 2023 @01:33PM
(#34549834)
In my computer science undergrad program, we watched and discussed a movie about Joseph Weizenbaum and Ray Kurzweil. I still vividly remember that discussion.
At the time, it seemed obvious to me that Kurzweil got it right and Weizenbaum got it wrong; that clearly, ELIZA was a red herring; that people wouldn't be fooled that easily; that the first AI to truly pass the Turing test would be the real thing (and we'd definitely see one, and with that the Singularity, in my lifetime).
Re:“Computers enable fantasies” – the continued relevance of...
by grantcas on Friday January 27, 2023 @02:18PM
(#34550431)
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
Re:“Computers enable fantasies” – the continued relevance of...
by FreakLegion on Friday January 27, 2023 @10:12PM
(#34554749)
If you look at their comment history you'll see this account exclusively regurgitates this same text, over and over. It's the entirety of their comment history.
Re:“Computers enable fantasies” – the continued relevance of...
by ccvannorman on Friday January 27, 2023 @07:50PM
(#34553878)
> “On the one hand the computer makes it possible in principle to live in a world of plenty for everyone, on the other hand we are well on our way to using it to create a world of suffering and chaos. Paradoxical, no?”
Um, what?
I use a computer and it 100% is responsbile for my "world of plenty."
If you're using a computer and your life sucks, you're using it wrong. That's not the computer's fault, and one would do well to abstain from making blanket judgements about the human-computer relationship as if the idea of a computer is to blame.
Re:“Computers enable fantasies” – the continued relevance of...
by 8bitsrule on Saturday January 28, 2023 @03:42AM
(#34556182)
"And if you are being fooled, it is worth considering who it is that is trying to fool you…"
Well worth reading. You'd think we'd be more cautious after learning what the oil companies didn't share with us. And how much we've invested in that we now have to discard.
Does GPT provide easily-read citations for each of its assertions? What steps does it take to ensure it's balanced, ethical? Doing that stuff well takes time.
> Writing of the enthusiastic embrace of a fully computerized world, Weizenbaum grumbled, “These people see the technical apparatus underlying Orwell’s 1984 and, like children on seeing the beach, they run for it”
> a point to which Weizenbaum added “I wish it were their private excursion, but they demand that we all come along.”
It's not really clear to me if this article is directed at scientists and engineers getting carried away or the general public. If the former, yes they should know better, but the general public is mostly not rational, contrary to what I used to think about the reasoning person. Many people attribute thoughts and feeling to their pets that they don't have. They think crystals have special powers. It's only natural that they will think AI have consciousness. I think as scientists and engineers we have a responsibility to prevent our AI creations from telling people to drink the poisoned Kool Aid or eat glass. Yann LeCun said they shut down Meta's chat AI because it told people that suicide had advantages and one can eat glass. Google is similarly cautious. OpenAI came along and let the cat out of the bag. I wonder if Microsoft has thought through what might happen if they hook it to Bing and it starts telling people crazy things. All it took was a couple of posts on 4chan to get the Q Anon movement off the ground.
That seems just a matter of time until they would be doing this rightly so.
Then some time later as we figure out how consciousness works in the computer - as it is much easier to inspect than a human - and compare the findings with humans to see if they indeed work in a similar way, we will all be a bit disappointed that the solution is much more mundane than we thought or hoped.
How do you know that? To be clear, I totally agree that ChatGPT is not yet there and I also think that scaling it up will not get us there, but I think the gap might be much smaller than most people think.
The way we think is often divided into two categories, for simple things like 1 + 1 you just know the answer, for complicated things things like 13 * 47 we really have to think and reason in steps. ChatGPT seems to do pretty well in the first category but it is not really capable of doing things in the second category. On the other hand I have seen examples of people talking ChatGPT through a reasoning process to arrive at the correct answer for something that it got initially wrong, for example ROT13 encoding some text.
So what if we stuff two copies of ChatGPT into a black box and instead of just spitting out what ever ChatGPT spits out, we let the two copies have some inner dialog? I don't think it is perfectly obvious how one would do this or what the result would be, but I think ChatGPT has enough basic knowledge that there is at least a chance that one could get it to reason in a step by step fashion.
But that also doesn't imply we should suspect any particular AI of being conscious just because AI in general could be conscious in principle. We don't expect worms to be conscious just because animals can be.
Now I'm not sure it matters. If people imbue their devices with intelligence and emotions then that's what they perceive. I've since learned about Tulpas. Some people have imaginary friends that seem very real to them. People will likely move their Tulpas out of their minds and into their AI's.
https://en.wikipedia.org/wiki/Tulpa
People on reddit are already talking about Tulpa's and AI
https://old.reddit.com/r/Tulpa/
but 'written explanations' only really work for done and completed things, or sometimes for future things which we want to repeat; i.e. which are based on things we've done before
I think that is a core part of the difficulty coming up in these discussions.
I hope you realize how silly that sounds.
No, I'm sure it looks like an airplane is flying and being onboard gives the right sensation but until we solve the hard problem of turbulent flows we really don't know if it's flying or not.
yea, let's leave this job to evil and ignorant humans...
https://www.perplexity.ai/ already did this and its really good and not this dystopian thing you luddites keep imagining.
Successful software solves a problem correctly and efficiently. It doesn't matter that GCC doesn't compile code like I would if I had to do that by hand, you still end up with compiled code, which was the original problem to solve. It doesn't matter that the sort() routine doesn't sort numbers like I would, it still does.
Playing the imitation game for its own sake is kind of silly.
The "illusion" lies in the attribution to a machine of capabilities it does not have.
This is true, but I think there is also a fundamental reason computers cannot understand us. If I tell a future AGI "I am sick" or "I am happy", the AGI may have a conceptual understanding of what I mean, but cannot empathize with my condition. Many human experiences are intimately tied to our bodies and our hormonal systems. How do you explain a concept like "pain" to an intelligence that has no nerves?
this reflects a failure to grasp the changing nature of agency in a world brimming with hybrid intelligence
we would perhaps like to think we and we alone "drive" history, and I agree with the author that we ought to try to do so, but technology carries its own inertia and shapes human activity in profound ways
AGI exists, it just has humans in the loop. paperclip maximizers exist, they're made of people and code. we are steered daily by alien incentive structures, which accrete as we interact with our implements and artifacts.
technology carves ruts in the mind and in the world, not to mention the burgeoning agency we continue to donate to it directly
As for "distinguishing human and procedural decisionmaking" I would define it as basically "human" decisionmaking is when there is some procedure but the human will disregard procedure based on some model of the "intent." You have to trust the human for this to work. So there's a fourth question, which is under what circumstances you could trust a computer to correctly model your intent. But then there's a fifth question, which is "whose intent?" But whose intentions the process serves isn't really intrinsically tied to whether humans or computers are running the procedure.
Another thing is that Weizenbaum and the author seem to take it for granted that human decisionmaking is literally magic and a computer is incapable of that kind of magic. But it's unclear and I think it's an important thing to state outright.
Which is itself separate from the question of whether or not Mark Zuckerberg is being kind of a dick.
The article isn't bad and the reference is worth reading, but it's impossible to engage with this much material at the same time as though it's one conversation. It's poor intellectual discipline, I'm sorry.
What price is an airline ticket on the first day of a global pandemic? No one knows, and there isn't anything to provide a "procedural system" in order for it to model what our "intention" would be in those cases.
This seems obvious with a pandemic, but it is common place. Every day is a genuinely novel occurence, and counts only as a repetition of some superficial patterns in very very limited respects.
When this novelty becomes relevant to a problem can never, (obviously), be pre-stated because we don't yet know what that novelty is.
There is simply no substitute for being-in-the-world as animals are, that is: moving around, communicating, acting, reacting and participating. Growing, learning, reproducing, and problem-solving in direct causal contact with reality.
This isn't magic, but there is no "procedural model" of it; nor should we expect there to be. There aren't procedural models of almost any phenomenon.
What we can build a computer to do is "almost nothing", and we shouldnt expect the trivial problem space given by this to exhaust what we actually need machines to do.
You're stating that a large part of human behavior is reacting to the environment presented to them via sensory input/output mechanisms. What you have not stated is why a machine entity cannot do the same things?
Nothing, because any procedure that exists was necessarily defined by humans. If the procedure is implemented by a computer, at most that provides humans an excuse for choosing not to disregard an outcome it produces.
> Another thing is that Weizenbaum and the author seem to take it for granted that human decisionmaking is literally magic
I can't imagine where this idea is coming from. The thrust of the argument is not that human decisions can't be modeled, but that omitting the question of whether they should be modeled, and those models granted (what is necessarily the pretense of) authority to make decisions in human stead, evades the same sort of responsibility that a civil engineer has to ensure that his bridge design can withstand the mechanical loads that it will need to bear in use.
ChatGPT will change the world! Sure, it already has. If we pay attention to the outcomes and see evidence of how it's changing the world.. it's mostly people doing the changing with their desires, beliefs, and politics. Same as always. The system itself is nothing but an automaton.
But aren't we all automatons too, building our self awareness around models of the word? Ah the AI version of the, "You can't prove that it's not consciousness!" Well we can as many scientists studying consciousness in humans and animals can attest: it's complicated, but we're getting better at understanding what it is. ChatGPT ain't it.
Definitely worth looking at the motivations of the people building the technology itself.
This speaks to my view exactly, and I haven't seen it put better.
The latent (imv) schizophrenic impulse to impart consciousness to any-old-object in the world is here being exploited by peddlers (often charlatans) of technology. This makes the "confirmation bias" here severe and dangerous.
People don't just want to believe, their animal psychology primes them for (over-)imparting consciousness. This is a specific cognitive vulnerability that is highly pernicious.
Even, trivially, consider "deresponsibilization", that process by which people take whatever-the-machine-says to be "The Answer". Why? Well, the machine "must know".
Less trivially, moral bias: "presumably" the machine is "objective" and so is morally superior.
How often do we hear this? Too much.
Have any examples of charlatans peddling "conscious" AIs?
I think we're seeing this playing out in discussions about generative AI with people justifying certain behaviors regarding the use of artist's data in training models by comparing the program to a human going through the process of "learning" and "gaining inspiration" from the work of others. Some people seem fully convinced that what these programs are doing is equivalent to human behavior to a degree that would qualify the program to receive human-like considerations, i.e. treating the software like an entity instead of just a program/tool to be wielded by the end-user.
I think this is one of the most pernicious issues, and people seem to have a hard time recognizing the trap they've fallen into.
I think this is also an outcome of this software being sufficiently complex to be too hard to understand without investing time and effort in understanding it.
I think this results in the mis-attribution of intelligence when instead it's just a really clever piece of software. But the casual commenter cannot estimate the gap between "pretty clever" and "so complex that it's actual intelligence".
So, let me set the first trap for you. Give me a logically/mathematically defined definition of intelligence and one of consciousness. Unfortunately after decades of back and forth on this we typically end up with answers that fall out of science and into the "I'll know it when I see it territory".
If you are going to describe machine intelligence/consciousness your definition must be able to cover intelligent behaviors we see from single cell life to the massive complexity we see in humans. Attempting to handwave and say "it's not as complex as humans therefore it's not intelligent/conscious" is a complete and total failure from my point of view.
How is your trap functioning? Are you saying that because we cannot precisely define consciousness, we cannot make any conclusions about it at all? Or that conversely, we should assume everything is conscious?
The scientific community continues to operate without certainty in many major areas that are deeply consequential, but that does not prevent us from exploring the problem space with the tools we do have.
There is plenty that we do know about the subjective experience of consciousness in biological creatures, human and otherwise. For it to even matter that a computer might be conscious is a construction of our subjective reality and our intuitions about why it is meaningful that something is conscious.
We've studied the relative complexity of thousands of species and understand enough to know that some species are a lot closer to humans than others.
But all of that is a giant deviation, and I'd argue has no bearing on the core point: the entire notion of copyright and the legal system it is built on are deeply, intrinsically, inherently human, and originate from the framework of human subjective consciousness, individual and collective. If the fact that the software is AI has any bearing on whether or not the unlicensed use of artist's content is acceptable or not, it must imply some elevated status of the software above ordinary software. It is that elevation that must be explained, and the explanations thus far have all been some form of "it's learning like a human".
Even if we were talking about a fully conscious AGI right now, we'd still need to have a conversation about what its consciousness means, and in what ways it is or is not compatible with human consciousness. Before that, we'd need to have a conversation about the ethics of commanding conscious AIs to do our bidding, but I digress.
We know not all consciousness is the same because we know to avoid grizzly bears.
Unless you're making an argument for Panpsychism, in which case this is an entirely different conversation :)
That isnt an endorsement of the argument, my point is that you dont have to believe the black box has any independent intelligence to draw an analogy between what it does and what we already allow.
Setting aside the bandwidth and compute issues for a moment, if Stable Diffusion was a tool that when prompted, downloaded and ingested 2.4 billion images (regardless of license), ran some really complex algorithms on them, and then spit out a derivative result - in other words, take out the AI - I think people would view the tool very differently.
At some point along the way, it seems people jump to a belief that because of <some component / step in the process>, this is no longer just a computer program that scraped the entire Internet without asking.
Wait, what? Who is doing this??
Some went further to claim that we don't know enough about consciousness to make any judgements about the nature of this software. As if somehow, this lack of knowledge implies the system must be conscious by default. It's a weird line of argument, but points to how strongly people feel pulled to confer consciousness on things they do not understand.
Maybe the panpsychists have it right, but that's an argument to be had at a much broader level and unrelated to AI, but that seems akin to living your life according to Pascal's wager. And in the case of AI, the cost of treating it with this reference could be great, and the potential for abuse greater.
After all, panpsychism has not been defeated as a philosophical standpoint [0]. Perhaps any-old-object in the world is actually conscius, just not all of us can detect it.
That the impulse to see consciousness in a thing is being used by charlatan is unfortunate, but has no bearing on the actual truth of whether the thing actually is conscious.
[0]https://plato.stanford.edu/entries/panpsychism/
There ought be a name for this type of psychological manipulation, but I don't know it. It is universal in a certain type of "motivational scam".
Crypto Scammer: Isn't it bad that finance is so centralised?
Crypto Scammer: Let me sell you a decentralized database!
Sceptic: But that isn't the same sense of decentarlization?
Scammer: Ah! But doesnt power flow from centralization of infrastructure?
Sceptic:... maybe, maybe not. Hey, why do you have your hand in my pocket?
Here, whether "consciousness" is on a continuum across all physical objects, or whether only a subset of objects have it, is kinda irrelevant:
The machine doesnt care about you, has no intentions, has nothing it wishes to say, and doesnt know anything about your or anything else. Its text generation is only a reflection of you, and the average of the superficial data of everyone else around you.
In sum:
Don't take philosophy classes from sales people.
I'm with you. The scammer doesn't care.
> The machine doesnt care about you
Uh, no. It's irrelevant to the scam. But now you're guessing at the internal feelings of an alien consciousness.
How do I know if I am not just a reflection of my environment, and my behaviour an average of superficial data of everyone around me?
Prove me conscious, and I'll start taking you seriously.
You realise, this is the same fallacy?
You've shifted the burden of proof on to some bizarre philosophical ground so that you dont have to defend your trick magic box.
A priori, by common sense, you're conscious. A priori, by common sense, a machine is not. There lie the default burdens of proof. From this the sales person must proceed to convince me, not vice versa.
I don't have to defend the box. I defend the right to say that we don't know in the absence of facts. Common sense isn't facts, nor is it a priori.
It has quite a bit of bearing, given that the reason we have discussions of consciousness at all in our own perception (real or imagined) of our own awareness being conscious. This arises from the very same brain / self that is innately driven to pareidolia. Literally the mechanisms that let us perceive consciousness arising (or project consciousness onto) other humans are the same ones responsible for pareidolia.
We've seen this in deities for as long as we've been around, royalties then took the crown, later subsumed by governments, then the daft notion of non-human personalities called companies. Now, aproximate intelligence systems.
We humans are credulous
- AGI is possible.
- AGI is possible without recreating qualia / an experiencer in some form.
It goes into the idea of intelligence without consciousness very well.
Why should there be an experiencer? Is that necessary or even optimal for intelligence?
Even when the Great Leader is unmasked as a fool or a deliberate con, people often still continue to believe. Sometimes they respond to the revelation by doubling down.
Adjusting exploited entities by probability of exploitation and adjusting advantaged entities by probability of advantage also leads me to radically different conclusions. For example, Ray Dalio is one advocate for using AI to supplement decisions. He is also exceedingly rich and a large part of his fortune was made with the help of investment decisions predicated on AI reasoning combined with human reasoning. Even though on average people who try to benefit from AI might not, the average benefit can still be massively positive, because the outcome distribution isn't uniform. Losing means losing a small fraction of a small amount, but winning means gaining an extreme amount to such an extent that it pays for the bad bets. This appears to often happen in practice - see the venture industries economics for evidence of this - and I don't see why it shouldn't apply to AI, which is something present in most unicorns that people often point to when discussing this concept.
gp was talking about humans' attribution of consciousness to inanimate objects, not other humans.
Nearly all religions apart from the more abstract big three would be counterexamples to your example. Dryads and naiads, all kinds of shrines and oracles and talismans...
I don't think you will be confident enough to put disagreement with my epistemics where your money is. However, to encourage you to do so, I will allow you to instead opt to donate the money to a charity of your choosing rather than giving it to me.
> humans' attribution of consciousness to inanimate objects
and in that context, I think the text you wrote is off topic. But StockFish is good at calculating chess moves yes. And a counting stick is good at counting numbers, is it also alive?
Might help to talk about decision theory rather than game theory. Check this:
P(heads=0.6), reward = 1. Notice that an agent optimizing expected value will recommend always heads for every estimate of P(heads) > 0.5. Here the observed probabilities can be 0.6 and the estimate could be 0.9 and even though there is error in the estimate of expected value it doesn't translate to a difference in expected value of the recommended strategy given that estimate. This is worth dwelling on with regard to his argument, because it helps to notice that his argument for why exploitation exists isn't even valid for arguing that error expected utility in simpler decisions problems exists. If he doesn't even have good argumentative structure for a much much less complicated decision problem, why is introducing exploitability, which is done with respect to multiple agents, going to be correct?
Probably still exploitable, but his argument for why is trash and he doesn't at all establish that it is particularly exploitable relative to other situations.
That is why I can give an example of a superhuman agent that always errors in this way, but isn't exploitable. His argument isn't valid.
My point doesn't just extend to Stockfish. It is far more general. Overestimation generally produces unexploitable strategies. This same process shows up in many AI which incorporate game theoretic reasoning, from Pluribus to AlphaZero, it is a very common attribute of our superhuman systems. In very sharp contrast, the systems which are capable of underestimating their opponent do so and can be exploited as a consequence. They are often much worse in general, partly because the reasoning process isn't particularly solid, but also because estimation of opponent beliefs is a lot more computationally intensive. So they think more slowly.
But you are kind of dodging the spirit of my point: if you disagree with me, lets construct a wager.
You are defending the position that humans are unusually exploitable. So how much would you be willing to risk on that claim? How much are you willing to bet you can exploit a particular human - like, for example, me. At a particular time, like for example, your next comment. How much money do you expect to extract from me with your comment using this 'exploit'? I wouldn't be greatly disadvantaged by you taking even $1,000 so lets make the minimum amount you must extract to qualify as exploiting me as $1,000. Can you do this? Can you exploit this hole right now so that I can see that you are right and admit that you knew better than me about the presence of this exploitable hole that you can see but which I do not see?
If you really believed what he was saying, you should be willing to do this. A magician would really believe they could exploit errors in human visual perception for example. it isn't a bet for them. It is their career.
You seem to be trying to make his claim be about attribution rates, an exceedingly lesser claim, wherein your proposal that I am out of context would have some merit. His claim was far far broader than that.
You need to calculate utilities, not attribution rates, in order to be able to make the claims he is trying to make. If you knew this, but still think I am wrong, then you are making a blunder. The context of this task is one of imperfect information. This means you need to do the evaluation relative to information state, not state. For several reasons, failure to do this is going to lead you to misleading and inaccurate estimation. Reach probabilities are one reason, because you have to scale the outcomes by the probability of getting to that information state. Another is that because you start in an information state, not with full knowledge of the result of the classification, you can't actually judge as if you knew which classification situation you were in. So all relevant possibilities have to be considered. For example, one I left out is the false negatives case: a person killed by a camouflaged solider, because they misidentified it as being inanimate, when it wasn't.
Sticks are a bad example to use, because they come from living objects. So, yes, they can indeed be alive. In this case, alive, but probably rapidly dying or having become deceased in the past. In some cases you can attach the stick back to its original home and it will get to live. I feel like this in this you intended to provide an example which doesn't contradict your point. So it might make more sense if we talk about a counting rock rather than a counting stick. So I'm going to try and make your argument stronger before I address it:
> And a counting rock is good at counting numbers, is it also alive?
Obviously the answer here is that it is not alive. However, this does not at all establish the OPs claim of exploit-ability. The question you must ask to establish your claim is "And a counting rock is good at counting numbers, assuming someone ends up thinking it is alive, does it follow that others can exploit that person?" Or maybe an even more fair question would be "And a counting rock is good at counting numbers, assuming you observe someone to appear to think that the rock is alive, can you exploit them?"
The flavor of his assertion is very different than the flavor of your question.
He is talking about the concept of exploitability. I'm addressing that context by expanding it to focus on other relevant parts of the classification task - focusing on the false positives screens off information that is critical for estimating the exploitability of a strategy. When you include other things, like true positives and true negatives, you get a very different expected outcome. Notable, you can't not do this and be reasoning correctly. One of the interesting and counterintuitive aspects of decision making under imperfect information which differentiates it with decision making under perfect information is that decision points influence your expected value calculations even when you are not in that situation. For example, if you are investing in a startup A and it is going to fail, that doesn't mean you shouldn't invest in that startup, because the results of investing in startup B, startup C, and startup D all have an impact on whether you should invest in startup A. For good writing on this, see Paul Graham. For great writing on this, read research in game theoretic reasoning.
But all of this is pointless to talk about if we don't actually disagree. So... If you really really disagree with me, in my world, the best way for us to settle this is with a bet. I'll bet you $10,000 that you, as a human playing without the aid of software, will lose to a chess playing program of my choosing which has the property of reasoning about your moves with the expectation that you are a stronger player than you actually are. I think the contention that an agent which overestimates others being exploitable is false. A happy accident here is that if I'm wrong to be at all impressed with AI, you shouldn't just be able to win because of the "exploitable" tendency it has to assume its opponent is stronger than it should. You should also be able to be much better than it, because it isn't like AI is better than human in decision making, right?
So when you say:
> counterexamples to your example
That doesn't make any sense to me, for much the same reason that someone telling me -3 is a counterexample to the idea of five doesn't make sense to me.
It certainly influenced how I think about this topic.
Perhaps we are finally thinking that AI is getting somewhere precisely because we don't understand how neural nets really work yet. It keeps the magic alive long enough for us to realize that progress really is being made.
Which might affect the way we look at technologies that try to model it, or at least use similar approaches - i.e. we're primed to treat them as more "magic" than they really are, or else to invent some reason why we think ourselves to be different.
One thing I have thought about over the years more and more, in this general vein is that it is sometimes to think about "computing technology" as being an evolution less as a specific thing itself and more of a broader "automation technology" which emerges most dramatically with the arrival of the industrial revolution (though there were aspects of it before that.)
Thinking with a kind of materialist analysis, we can see computing but even (more broadly) higher level mathematics, logic, etc. as an intellectual effort to chip away at labour costs & the percentage of human effort involved in production/work/business. It's a deeply obscured profit motive.
The ideological blindness described in this article I think fits neatly into that: we are all so enamoured with this computing thing, this process, the puzzle and the end results that we often fail to see that it is not some magical force operating under its own motive, with its own natural inevitable destination and we often do not see that there's likely a deeper force underlying all of this... human efforts, labour, driven by profit and hunger.
Ultimately computing technology is capitalist automation technology. It reflects the needs of the capitalist market system, and the drive to efficiency that the market economy requires. The 'magic' we seem to pepper overtop is our own flourish.
(Maybe when I'm finally done with this software engineering career that has eaten my life, I will go back and finish my philosophy degree I walked away from in the 90s... and then I will write more coherently about this)
But the thing is.. how do they say it? "Money talks, bullshit walks?"
People follow what fills their stomach. And mechanization/automation/computation, it does that in spades. I mean... I dropped out of my philosophy major decades ago to go do the More Fun To Compute thing... My bank account shows it.
In any case... the first major uses of computers were for war. The next, for automation in commerce and finance. That's what computers are really for.
At the time, it seemed obvious to me that Kurzweil got it right and Weizenbaum got it wrong; that clearly, ELIZA was a red herring; that people wouldn't be fooled that easily; that the first AI to truly pass the Turing test would be the real thing (and we'd definitely see one, and with that the Singularity, in my lifetime).
I think about Weizenbaum a lot these days.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
Um, what?
I use a computer and it 100% is responsbile for my "world of plenty."
If you're using a computer and your life sucks, you're using it wrong. That's not the computer's fault, and one would do well to abstain from making blanket judgements about the human-computer relationship as if the idea of a computer is to blame.
Well worth reading. You'd think we'd be more cautious after learning what the oil companies didn't share with us. And how much we've invested in that we now have to discard.
Does GPT provide easily-read citations for each of its assertions? What steps does it take to ensure it's balanced, ethical? Doing that stuff well takes time.
Instant gratification? Danger Will Robinson.