Have any examples of charlatans peddling "conscious" AIs?
I think we're seeing this playing out in discussions about generative AI with people justifying certain behaviors regarding the use of artist's data in training models by comparing the program to a human going through the process of "learning" and "gaining inspiration" from the work of others. Some people seem fully convinced that what these programs are doing is equivalent to human behavior to a degree that would qualify the program to receive human-like considerations, i.e. treating the software like an entity instead of just a program/tool to be wielded by the end-user.
I think this is one of the most pernicious issues, and people seem to have a hard time recognizing the trap they've fallen into.
I think this is also an outcome of this software being sufficiently complex to be too hard to understand without investing time and effort in understanding it.
I think this results in the mis-attribution of intelligence when instead it's just a really clever piece of software. But the casual commenter cannot estimate the gap between "pretty clever" and "so complex that it's actual intelligence".
So, let me set the first trap for you. Give me a logically/mathematically defined definition of intelligence and one of consciousness. Unfortunately after decades of back and forth on this we typically end up with answers that fall out of science and into the "I'll know it when I see it territory".
If you are going to describe machine intelligence/consciousness your definition must be able to cover intelligent behaviors we see from single cell life to the massive complexity we see in humans. Attempting to handwave and say "it's not as complex as humans therefore it's not intelligent/conscious" is a complete and total failure from my point of view.
How is your trap functioning? Are you saying that because we cannot precisely define consciousness, we cannot make any conclusions about it at all? Or that conversely, we should assume everything is conscious?
The scientific community continues to operate without certainty in many major areas that are deeply consequential, but that does not prevent us from exploring the problem space with the tools we do have.
There is plenty that we do know about the subjective experience of consciousness in biological creatures, human and otherwise. For it to even matter that a computer might be conscious is a construction of our subjective reality and our intuitions about why it is meaningful that something is conscious.
We've studied the relative complexity of thousands of species and understand enough to know that some species are a lot closer to humans than others.
But all of that is a giant deviation, and I'd argue has no bearing on the core point: the entire notion of copyright and the legal system it is built on are deeply, intrinsically, inherently human, and originate from the framework of human subjective consciousness, individual and collective. If the fact that the software is AI has any bearing on whether or not the unlicensed use of artist's content is acceptable or not, it must imply some elevated status of the software above ordinary software. It is that elevation that must be explained, and the explanations thus far have all been some form of "it's learning like a human".
Even if we were talking about a fully conscious AGI right now, we'd still need to have a conversation about what its consciousness means, and in what ways it is or is not compatible with human consciousness. Before that, we'd need to have a conversation about the ethics of commanding conscious AIs to do our bidding, but I digress.
We know not all consciousness is the same because we know to avoid grizzly bears.
Unless you're making an argument for Panpsychism, in which case this is an entirely different conversation :)
That isnt an endorsement of the argument, my point is that you dont have to believe the black box has any independent intelligence to draw an analogy between what it does and what we already allow.
Setting aside the bandwidth and compute issues for a moment, if Stable Diffusion was a tool that when prompted, downloaded and ingested 2.4 billion images (regardless of license), ran some really complex algorithms on them, and then spit out a derivative result - in other words, take out the AI - I think people would view the tool very differently.
At some point along the way, it seems people jump to a belief that because of <some component / step in the process>, this is no longer just a computer program that scraped the entire Internet without asking.
Wait, what? Who is doing this??
Some went further to claim that we don't know enough about consciousness to make any judgements about the nature of this software. As if somehow, this lack of knowledge implies the system must be conscious by default. It's a weird line of argument, but points to how strongly people feel pulled to confer consciousness on things they do not understand.
Maybe the panpsychists have it right, but that's an argument to be had at a much broader level and unrelated to AI, but that seems akin to living your life according to Pascal's wager. And in the case of AI, the cost of treating it with this reference could be great, and the potential for abuse greater.
After all, panpsychism has not been defeated as a philosophical standpoint [0]. Perhaps any-old-object in the world is actually conscius, just not all of us can detect it.
That the impulse to see consciousness in a thing is being used by charlatan is unfortunate, but has no bearing on the actual truth of whether the thing actually is conscious.
There ought be a name for this type of psychological manipulation, but I don't know it. It is universal in a certain type of "motivational scam".
Crypto Scammer: Isn't it bad that finance is so centralised?
Crypto Scammer: Let me sell you a decentralized database!
Sceptic: But that isn't the same sense of decentarlization?
Scammer: Ah! But doesnt power flow from centralization of infrastructure?
Sceptic:... maybe, maybe not. Hey, why do you have your hand in my pocket?
Here, whether "consciousness" is on a continuum across all physical objects, or whether only a subset of objects have it, is kinda irrelevant:
The machine doesnt care about you, has no intentions, has nothing it wishes to say, and doesnt know anything about your or anything else. Its text generation is only a reflection of you, and the average of the superficial data of everyone else around you.
In sum:
Don't take philosophy classes from sales people.
I'm with you. The scammer doesn't care.
> The machine doesnt care about you
Uh, no. It's irrelevant to the scam. But now you're guessing at the internal feelings of an alien consciousness.
How do I know if I am not just a reflection of my environment, and my behaviour an average of superficial data of everyone around me?
Prove me conscious, and I'll start taking you seriously.
You realise, this is the same fallacy?
You've shifted the burden of proof on to some bizarre philosophical ground so that you dont have to defend your trick magic box.
A priori, by common sense, you're conscious. A priori, by common sense, a machine is not. There lie the default burdens of proof. From this the sales person must proceed to convince me, not vice versa.
I don't have to defend the box. I defend the right to say that we don't know in the absence of facts. Common sense isn't facts, nor is it a priori.
It has quite a bit of bearing, given that the reason we have discussions of consciousness at all in our own perception (real or imagined) of our own awareness being conscious. This arises from the very same brain / self that is innately driven to pareidolia. Literally the mechanisms that let us perceive consciousness arising (or project consciousness onto) other humans are the same ones responsible for pareidolia.
We've seen this in deities for as long as we've been around, royalties then took the crown, later subsumed by governments, then the daft notion of non-human personalities called companies. Now, aproximate intelligence systems.
We humans are credulous
- AGI is possible.
- AGI is possible without recreating qualia / an experiencer in some form.
It goes into the idea of intelligence without consciousness very well.
Why should there be an experiencer? Is that necessary or even optimal for intelligence?
Even when the Great Leader is unmasked as a fool or a deliberate con, people often still continue to believe. Sometimes they respond to the revelation by doubling down.
Adjusting exploited entities by probability of exploitation and adjusting advantaged entities by probability of advantage also leads me to radically different conclusions. For example, Ray Dalio is one advocate for using AI to supplement decisions. He is also exceedingly rich and a large part of his fortune was made with the help of investment decisions predicated on AI reasoning combined with human reasoning. Even though on average people who try to benefit from AI might not, the average benefit can still be massively positive, because the outcome distribution isn't uniform. Losing means losing a small fraction of a small amount, but winning means gaining an extreme amount to such an extent that it pays for the bad bets. This appears to often happen in practice - see the venture industries economics for evidence of this - and I don't see why it shouldn't apply to AI, which is something present in most unicorns that people often point to when discussing this concept.
gp was talking about humans' attribution of consciousness to inanimate objects, not other humans.
Nearly all religions apart from the more abstract big three would be counterexamples to your example. Dryads and naiads, all kinds of shrines and oracles and talismans...
I don't think you will be confident enough to put disagreement with my epistemics where your money is. However, to encourage you to do so, I will allow you to instead opt to donate the money to a charity of your choosing rather than giving it to me.
> humans' attribution of consciousness to inanimate objects
and in that context, I think the text you wrote is off topic. But StockFish is good at calculating chess moves yes. And a counting stick is good at counting numbers, is it also alive?
Might help to talk about decision theory rather than game theory. Check this:
P(heads=0.6), reward = 1. Notice that an agent optimizing expected value will recommend always heads for every estimate of P(heads) > 0.5. Here the observed probabilities can be 0.6 and the estimate could be 0.9 and even though there is error in the estimate of expected value it doesn't translate to a difference in expected value of the recommended strategy given that estimate. This is worth dwelling on with regard to his argument, because it helps to notice that his argument for why exploitation exists isn't even valid for arguing that error expected utility in simpler decisions problems exists. If he doesn't even have good argumentative structure for a much much less complicated decision problem, why is introducing exploitability, which is done with respect to multiple agents, going to be correct?
Probably still exploitable, but his argument for why is trash and he doesn't at all establish that it is particularly exploitable relative to other situations.
That is why I can give an example of a superhuman agent that always errors in this way, but isn't exploitable. His argument isn't valid.
My point doesn't just extend to Stockfish. It is far more general. Overestimation generally produces unexploitable strategies. This same process shows up in many AI which incorporate game theoretic reasoning, from Pluribus to AlphaZero, it is a very common attribute of our superhuman systems. In very sharp contrast, the systems which are capable of underestimating their opponent do so and can be exploited as a consequence. They are often much worse in general, partly because the reasoning process isn't particularly solid, but also because estimation of opponent beliefs is a lot more computationally intensive. So they think more slowly.
But you are kind of dodging the spirit of my point: if you disagree with me, lets construct a wager.
You are defending the position that humans are unusually exploitable. So how much would you be willing to risk on that claim? How much are you willing to bet you can exploit a particular human - like, for example, me. At a particular time, like for example, your next comment. How much money do you expect to extract from me with your comment using this 'exploit'? I wouldn't be greatly disadvantaged by you taking even $1,000 so lets make the minimum amount you must extract to qualify as exploiting me as $1,000. Can you do this? Can you exploit this hole right now so that I can see that you are right and admit that you knew better than me about the presence of this exploitable hole that you can see but which I do not see?
If you really believed what he was saying, you should be willing to do this. A magician would really believe they could exploit errors in human visual perception for example. it isn't a bet for them. It is their career.
You seem to be trying to make his claim be about attribution rates, an exceedingly lesser claim, wherein your proposal that I am out of context would have some merit. His claim was far far broader than that.
You need to calculate utilities, not attribution rates, in order to be able to make the claims he is trying to make. If you knew this, but still think I am wrong, then you are making a blunder. The context of this task is one of imperfect information. This means you need to do the evaluation relative to information state, not state. For several reasons, failure to do this is going to lead you to misleading and inaccurate estimation. Reach probabilities are one reason, because you have to scale the outcomes by the probability of getting to that information state. Another is that because you start in an information state, not with full knowledge of the result of the classification, you can't actually judge as if you knew which classification situation you were in. So all relevant possibilities have to be considered. For example, one I left out is the false negatives case: a person killed by a camouflaged solider, because they misidentified it as being inanimate, when it wasn't.
Sticks are a bad example to use, because they come from living objects. So, yes, they can indeed be alive. In this case, alive, but probably rapidly dying or having become deceased in the past. In some cases you can attach the stick back to its original home and it will get to live. I feel like this in this you intended to provide an example which doesn't contradict your point. So it might make more sense if we talk about a counting rock rather than a counting stick. So I'm going to try and make your argument stronger before I address it:
> And a counting rock is good at counting numbers, is it also alive?
Obviously the answer here is that it is not alive. However, this does not at all establish the OPs claim of exploit-ability. The question you must ask to establish your claim is "And a counting rock is good at counting numbers, assuming someone ends up thinking it is alive, does it follow that others can exploit that person?" Or maybe an even more fair question would be "And a counting rock is good at counting numbers, assuming you observe someone to appear to think that the rock is alive, can you exploit them?"
The flavor of his assertion is very different than the flavor of your question.
He is talking about the concept of exploitability. I'm addressing that context by expanding it to focus on other relevant parts of the classification task - focusing on the false positives screens off information that is critical for estimating the exploitability of a strategy. When you include other things, like true positives and true negatives, you get a very different expected outcome. Notable, you can't not do this and be reasoning correctly. One of the interesting and counterintuitive aspects of decision making under imperfect information which differentiates it with decision making under perfect information is that decision points influence your expected value calculations even when you are not in that situation. For example, if you are investing in a startup A and it is going to fail, that doesn't mean you shouldn't invest in that startup, because the results of investing in startup B, startup C, and startup D all have an impact on whether you should invest in startup A. For good writing on this, see Paul Graham. For great writing on this, read research in game theoretic reasoning.
But all of this is pointless to talk about if we don't actually disagree. So... If you really really disagree with me, in my world, the best way for us to settle this is with a bet. I'll bet you $10,000 that you, as a human playing without the aid of software, will lose to a chess playing program of my choosing which has the property of reasoning about your moves with the expectation that you are a stronger player than you actually are. I think the contention that an agent which overestimates others being exploitable is false. A happy accident here is that if I'm wrong to be at all impressed with AI, you shouldn't just be able to win because of the "exploitable" tendency it has to assume its opponent is stronger than it should. You should also be able to be much better than it, because it isn't like AI is better than human in decision making, right?
So when you say:
> counterexamples to your example
That doesn't make any sense to me, for much the same reason that someone telling me -3 is a counterexample to the idea of five doesn't make sense to me.
This speaks to my view exactly, and I haven't seen it put better.
The latent (imv) schizophrenic impulse to impart consciousness to any-old-object in the world is here being exploited by peddlers (often charlatans) of technology. This makes the "confirmation bias" here severe and dangerous.
People don't just want to believe, their animal psychology primes them for (over-)imparting consciousness. This is a specific cognitive vulnerability that is highly pernicious.
Even, trivially, consider "deresponsibilization", that process by which people take whatever-the-machine-says to be "The Answer". Why? Well, the machine "must know".
Less trivially, moral bias: "presumably" the machine is "objective" and so is morally superior.
How often do we hear this? Too much.