News for hackers. Stuff that matters.
Re:Conversational AI will be used to Manipulate Us in real-time by haolez on Thursday February 23, 2023 @09:22AM (#34910908)
Maybe this could be solved by teaching philosophy and logic in our schools. No matter how smart a conversational AI can be, you can still frame its argument and decide on an outcome.

But I suspect that the governments won't want that. At least in Brazil, where I live, this was aggressively pushed back a few decades ago. I guess they want voters that can't think straight :)

Re:Conversational AI will be used to Manipulate Us in real-time by qsort on Thursday February 23, 2023 @09:41AM (#34911155)
> this could be solved by teaching philosophy

Plenty of places teach philosophy, including the school I went to in Italy. The only measurable effect is that people feel more self-important because they know the names of long dead people.

(Just to clarify this being the Internet: I don't mean philosophy is in itself useless, just that ambitious programs at the high-school level tend to fall flat unless the student has a lot of intrinsic motivation. Latin, computer science, art history, etc. mostly fall in the same category. A better approach is to stick to the basics and offer those as electives.)

> and logic

Logic done seriously requires a mathematical maturity most high-school students simply don't have. Asking someone to understand models of first-order logic without intuitively grokking what quantifiers are is like asking someone to understand Rust lifetimes without having ever seen malloc. It just doesn't work like that. You'd have better success with stuff like Olympiad problems, but even those require so much training it definitely isn't for everyone.

Re:Conversational AI will be used to Manipulate Us in real-time by denton-scratch on Thursday February 23, 2023 @11:30AM (#34912644)
> Logic done seriously requires a mathematical maturity most high-school students simply don't have.

I guess that "done seriously", that's true. But basic logic is like basic arithmetic; they teach that in primary schools. I don't see why they shouldn't teach elementary logic.

In one of the Narnia books, the old professor bemoans the fact that "they don't teach logic in schools any more".

Philosophy is another matter; I agree with the upthread poster who says that philosophy requires motivation. I don't think an introduction to philosophy is much use, except to help you decide whether you want to learn philosophy properly, e.g. a couple of one-hour classes, so kids at least know what the word means (I didn't know, until I started my philosophy degree).

Re:Conversational AI will be used to Manipulate Us in real-time by retrac on Thursday February 23, 2023 @09:34AM (#34911049)
I suspect even philosophers are susceptible to the usual social engineering tricks. One doesn't become immune to effects like preferring a speaker with an attractive face just from exposure to Kant. (Just one notion that occurred to me just now. Won't be long until these things can have a real time "video chat" with users.)
Re:Conversational AI will be used to Manipulate Us in real-time by maroonblazer on Thursday February 23, 2023 @09:42AM (#34911174)
One still has to evaluate an argument on its merits, regardless of the attractiveness of the interlocutor.

My hope is that conversational AI will sharpen up our critical thinking skills en masse. We need more people getting into the habit of asking "This sounds plausible, but is it actually true? How can I find out?"

Re:Conversational AI will be used to Manipulate Us in real-time by TuringTest on Thursday February 23, 2023 @10:03AM (#34911452)
> One still has to evaluate an argument on its merits, regardless of the attractiveness of the interlocutor.

True, but unfortunately, rational thought is heavily influenced by emotions.

You are much more likely to accept some cognitive dissonances if the reasoning is caused by someone you trust, like, or admire.

Heavy thinkers are not immune to that, they just produce much more elaborate justifications.

Re:Conversational AI will be used to Manipulate Us in real-time by sixothree on Thursday February 23, 2023 @09:43AM (#34911183)
Imagine being surrounded by hundreds of conversational voices.
Re:Conversational AI will be used to Manipulate Us in real-time by haolez on Thursday February 23, 2023 @11:37AM (#34912759)
Well, this would be hell even if they weren't intelligent. I think this is not the point, but thanks for the nightmare material.
Re:Conversational AI will be used to Manipulate Us in real-time by dghughes on Thursday February 23, 2023 @09:58AM (#34911386)
Just keep asking why. A human will get pissed off an "AI" won't.
Re:Conversational AI will be used to Manipulate Us in real-time by acegopher on Thursday February 23, 2023 @09:39AM (#34911113)
Perhaps we will need a Butlerian Jihad of some form
Re:Conversational AI will be used to Manipulate Us in real-time by mlatu on Thursday February 23, 2023 @09:40AM (#34911130)
i read Us as "plural U" ... as in universities .?.
Re:Conversational AI will be used to Manipulate Us in real-time by andrewfromx on Thursday February 23, 2023 @09:40AM (#34911137)
The movie wolf of wall st shows Jordan’s “straight line persuasion” technique. i.e. keep the human on a straight line to your end goal of a sale and when they deviate from the straight line correct them back on it. AI will be fantastic at this with no moral dilemma!
Re:Conversational AI will be used to Manipulate Us in real-time by Zurrrrr on Thursday February 23, 2023 @09:42AM (#34911161)
I don't see how to solve this. This is the same root cause that causes people to be 911 truthers or qanon or to vote for clearly nonsense candidates.

It's a lack of critical thinking, and it's an epidemic.

When you have people believing whatever they want on Facebook or whatever because someone makes minimal effort to photoshop something to look like a news post, of course that's something AI can take advantage of.

But it isn't a new problem, and the solution is the same.

But how do you educate a population that is aggressive and resistant to acquiring knowledge?

I'm just here to watch the band play as the ship goes down.

Re:Conversational AI will be used to Manipulate Us in real-time by fullshark on Thursday February 23, 2023 @09:52AM (#34911304)
Critical thinking requires time and effort. Offloading critical thinking is something every single person does in order to make sense of the world and make decisions, it's basically what trust is and it's necessary for the world to function.

The idea that we can just ignore this risk and allow critical thinking within individuals to handle it is not even wishful thinking, just shirking responsibility imo.

Re:Conversational AI will be used to Manipulate Us in real-time by TuringTest on Thursday February 23, 2023 @09:59AM (#34911395)
Also, critical thinking is a social phenomenon.

You make sense of the world based on the sources you trust, and that trust is created gregariously by learning to rely on what your peers are sharing as trustworthy. The time and effort needed is shared with the group, and the resulting wisdom of the crowds filters out the most outrageous messages from the outside, developing a 'groupthink' that establishes a consensus of what is "common sense" for the group.

However, someone with the power to broadcast messages to the whole group can overcome that group processing, and sway the whole group to a different way of thinking.

Re:Conversational AI will be used to Manipulate Us in real-time by Zurrrrr on Thursday February 23, 2023 @10:06AM (#34911485)
I've never seen anyone defend and make excuses for people not thinking critically.

Sure, trust is one thing, and we relax critical thinking....but trust among people we know has nothing to do with failures as large as thinking, say 5G causes covid. Trust has nothing to do with that really.

And I'm not ignoring the risk. I absolutely acknowledge it Trump almost got elected twice for chrissakes. I just don't see what can be done about it with a population resistant to thinking.

Re:Conversational AI will be used to Manipulate Us in real-time by wpietri on Thursday February 23, 2023 @10:47AM (#34912040)
I am a big fan of critical thinking, but shouting "think harder people" is just not a solution to anything, and neither is giving up. We can and should change our system to support critical thinking, and we can change it to demand much less critical thinking to make effective decisions.
Re:Conversational AI will be used to Manipulate Us in real-time by Zurrrrr on Thursday February 23, 2023 @11:00AM (#34912220)
It's not so much I'm shouting "think harder people", more lamenting that I think it's too late. The shouting should have been done about 30 years ago.

I'm cynical, but I don't really see a path forward for the US when half the population is so resistant to truth.

But absolutely, I agree with should be improving education and changing things so not every single thing should have to be analyzed to make a decision.

Re:Conversational AI will be used to Manipulate Us in real-time by pixl97 on Thursday February 23, 2023 @03:15PM (#34915860)
You're looking at such a short time frame, that it's not a surprise that this all seems like a new phenomenon to you.

Lets jump back to 90 years ago to Europe. Do you think the lead up to world war two was caused by a lack of critical thinking? Do you think if I sent you back to 1920 so you could howl into the wind about critical thinking if you believe you'd have made one iota of difference to the devastation that occurred?

I would like you to give me a power, that is to make critical thinking an uninterruptable process in your mind, and also have the ability to spam the shit out of you with things that you should probably be thinking critically about anyway. You would be a wreck in days. The number of things you take for granted about the world around you, things society and the social contract take care of silently and with little notice would now be things you have to think about. You would have to back process that entire queue of completely dumb shit we don't think critically about but likely should. You wouldn't get anything else done.

Instead you, me, and Mr McGhee push those things out of our mind because we have work do to, relationships to work on, food to eat, stress to relieve, new things to learn, etc. Humans require a lot of time doing a lot of illogical human crap to remain sane, and not set around being machines.

Re:Conversational AI will be used to Manipulate Us in real-time by Zurrrrr on Thursday February 23, 2023 @03:34PM (#34916102)
Where did I ver say anything that indicated I thought it was a new phenomenon or that I was limiting what I was describing to a short time frame?

That aside, no idea what you are arguing here. It seems to be that because critical thinking won't solve every single problem it's worthless?

Re:Conversational AI will be used to Manipulate Us in real-time by pixl97 on Thursday February 23, 2023 @04:17PM (#34916579)
You're very close to a breakthrough there... The first thing you have to accept is that 'critical thinking' is not a thing. There is only thinking. When you have that realization it is much easier to see when a person gets themselves in a situation that requires nonbiased, factual, information based processing to get out of they are exceptionally unlikely to do it as they lack the nonbiased factual data in the first place and likely have no realization it even exists.

And that's not even the hard part. No, that's the easy part.

The hard part is when there is a set of biased agents that keep feeding you bad information that keep you away from that thinking revolution that would allow you to escape the trap. Friends, politics, advertizing, these are all ever present agents that bias you to things like societal norms, to acceptable behaviors, that tell you what to think and not think about. And these are things that you also spread to other people. It's just part of being a society, a civilization.

Re:Conversational AI will be used to Manipulate Us in real-time by Zurrrrr on Thursday February 23, 2023 @04:24PM (#34916674)
I'm guessing from your username you were born in 97. How are you that young and arrogant and condescending lol.

There's no 'breakthrough' I'm close to or need to have. I said everything I needed to say and it remains accurate. You said a bunch of nonsense based on your own assumptions, and you're acting like you're trying to teach me something or some bullshit.

The only thing you continue to prove with your comments is that you yourself could really benefit from learning how to think critically before you reply. I won't be replying again to you, it's simply not worth the effort.

Cheers.

Re:Conversational AI will be used to Manipulate Us in real-time by pixl97 on Thursday February 23, 2023 @05:09PM (#34917300)
Poor assumption, that was the year I picked this particular username, far far older than that, and seemingly very agist on your part too.

If any problem involves "Everybody all of a sudden doing the right thing" the problem is not solvable if that is the only solution.

May we both live in interesting times.

Re:Conversational AI will be used to Manipulate Us in real-time by finnthehuman on Thursday February 23, 2023 @10:26AM (#34911766)
> It's a lack of critical thinking, and it's an epidemic.

It's because the non-AI text generators everyone spends too much time listening to (jounos, "content creators") rely on an audience that doesn't engage with their output critically.

Re:Conversational AI will be used to Manipulate Us in real-time by runsWphotons on Thursday February 23, 2023 @10:27AM (#34911781)
add in that even the real news is often lies or at least some kind of manipulation...
Re:Conversational AI will be used to Manipulate Us in real-time by zpeti on Thursday February 23, 2023 @10:34AM (#34911877)
Whatever political side you are in, it's extremely naive to assume the other side is making choices because of stupidity. Yes there are stupid people and conspiracy theorists on the "other side", but trust me, at the lower sophisticated end of your political side there are also really stupid ideas that people believe in.

We all have our reasons for believing stupid stuff, and voting for candidates that are "clearly nonsense".

Re:Conversational AI will be used to Manipulate Us in real-time by Zurrrrr on Thursday February 23, 2023 @10:58AM (#34912196)
Yes, there are idiots on both sides, but that's like Trump trying to equate racists with non-racists by saying there are good people on both sides.

It's only one side of the political spectrum that as a significant portion of people who think a Kenedy will return from the dead, that you can cure covid by 'putting the light inside', that 5G causes covid, etc etc etc.

The amount of ignorance isn't equally distributed at all, and it's silly to try and claim otherwise.

Re:Conversational AI will be used to Manipulate Us in real-time by zpeti on Thursday February 23, 2023 @03:26PM (#34915992)
As someone who grew in both eastern europe and the west, this view you have is quite wrong. The amount of stupidity on the left about "equality" and where it leads is very large, a lot of people believe it, even though it leads to more murderous places than right wing extremism ever did.

Contrary to you I don't believe these people are stupid, I just think the utopian vision of equality is a very potent one, but it also happens to be the most destructive path paved with good intentions.

Both sides have a lot of stupid ideas at the extremes, and there aren't less or more people who believe them, it just depends on how willing you are to look objectively.

Re:Conversational AI will be used to Manipulate Us in real-time by Zurrrrr on Thursday February 23, 2023 @03:37PM (#34916145)
There's a lot of people who grew up in the region you did who come to the weat and are suspicious and downright dismissive of many leftist ideals. That's a shame. Your experiences don't invalidate their talking points and it is still the most logical things to strive for if we want the US (or really any) country to do better.

One side is trying to force crayons as coins into a washing machine (that doesn't have a payment mechanism) to pay for laundry, the other is being fanciful and naive in some of their goals for a better tomorrow.

The two sides are not alike.

Re:Conversational AI will be used to Manipulate Us in real-time by zpeti on Friday February 24, 2023 @02:03AM (#34921742)
> and it is still the most logical things to strive for if we want the US (or really any) country to do better.

Actually, logic would dictate you want people to become wealthy, and inequality doesn’t matter. Usually though when societies create wealth they also create inequality, so while everyone does better, the rich do even better. People like yourself can’t see how this is a good thing, because you see equality is moral, except the only way you get equality is by pulling people down. This is the lesson of history, which many don’t understand. I’ve seen this at work in so many instances, for example the education system in the UK. The obsession with equality has not meant underperformers are lifted up, but all elite training is destroyed. Except then overall you just lower the level of everyone.

You need to get beyond your resentfulness of people doing well. Capitalism (and freedom) isn’t perfect, many people exploit it, and it creates inequality, but it’s much better for everyone than anything else. And you simply don’t have examples today or in history of anything different (and Nordic countries are still capitalist, btw).

Re:Conversational AI will be used to Manipulate Us in real-time by Zurrrrr on Friday February 24, 2023 @02:42AM (#34922016)
I'm not against capitalism, I just think there should be more regulation (like the Nordic countries).

Like I said, you are making a ton of assumptions out of fear and misinformation.

Re:Conversational AI will be used to Manipulate Us in real-time by kthejoker2 on Thursday February 23, 2023 @09:47AM (#34911230)
It's sad how far we've strayed from Bellamy's Looking Backwards style utopias and straight into Orwellian dystopias.

I predict the rise of more seasteaders, Oneida style communes, digital retreats, maybe even extreme cosplay like The Village ... a ton of people are just going to nope out of this hellscape.

Maybe the metaverse is onto something...

Re:Conversational AI will be used to Manipulate Us in real-time by pessimizer on Thursday February 23, 2023 @10:00AM (#34911411)
> I predict the rise of more seasteaders, Oneida style communes, digital retreats, maybe even extreme cosplay like The Village ... a ton of people are just going to nope out of this hellscape.

Wealthy people always do. We should worry about people who won't always have options.

Re:Conversational AI will be used to Manipulate Us in real-time by nathan_compton on Thursday February 23, 2023 @10:20AM (#34911668)
The metaverse is exactly the wrong place to flee to if you want to avoid AI that are native to the digital world.
Re:Conversational AI will be used to Manipulate Us in real-time by supriyo-biswas on Thursday February 23, 2023 @10:30AM (#34911819)
Meta is experimenting with LLMs and AI video generation precisely because of their applications in the meta verse to keep you hooked.
Re:Conversational AI will be used to Manipulate Us in real-time by throwaway4aday on Thursday February 23, 2023 @10:20AM (#34911671)
Frankly, Bellamy's utopia still sounds miserable and far less believable than Orwell's or even Huxley's. The organization of society in Bellamy is laughably naive of both human nature and economics. Honestly it reads like a children's book more than an achievable goal or prediction of things to come.
Re:Conversational AI will be used to Manipulate Us in real-time by wpietri on Thursday February 23, 2023 @10:42AM (#34911959)
One of the frequent critiques of older science fiction works is that they failed to predict the future, or that they seem unrealistic in light of what we now know. I think that misunderstands the point of science fiction.
Re:Conversational AI will be used to Manipulate Us in real-time by throwaway4aday on Friday February 24, 2023 @08:59AM (#34924541)
That's not what I'm accusing Bellamy of. Others have pointed out that certain aspects of his book do resemble things we have now. My critique is of his portrayal of the social and economic structure as well as the interpersonal dealings that come along with such systems. If you want specifics, he thought that the problem of labour could be solved by simply drafting everyone into a national labour army for 24 years but at the same time allow them to just do whatever they want and somehow that would result in perfect harmony. Those are two completely contradictory thoughts, you either believe people can self-organize and determine for themselves what field they should be working in and what work they should be doing or you believe that we need a strongly regimented society where everyone is legally obligated to fulfill duties at the behest of the state. He also doesn't consider any second order effects of this. What happens to immigration and emigration? If you're drafted you can't move to another country, so you've got two periods when you can move: as a kid or once you're retired (and it's not so clear that it would be possible once you retire since you're still technically on call according to Bellamy). That means no families can move since your child bearing and rearing years are owned by the state. So it's either you move by yourself as a kid or only once you're fully retired and not liable to perform any work for the state. He also thinks that everyone is just ok with this because it is expected of them, even in his day he should have seem the absurdity of this. It's all just socialist fantasizing with no grounding in reality.

The point of science fiction is to create a believable vision of the future, even if it is fantastical. You know good writing when you see it because you can suspend your disbelief until the end. If you get pulled out of the author's vision multiple times per paragraph you know it's crap.

Re:Conversational AI will be used to Manipulate Us in real-time by wpietri on Friday February 24, 2023 @10:59AM (#34925816)
> The point of science fiction is to create a believable vision of the future

It isn't, but even running with that, he did. It was an enormously popular book at the time, and fits in with a lot of other utopian socialist visions of that era.

I understand that it doesn't fit with your personal ideologies and beliefs more than century later. But you were not his audience. A book with the same goals written today would be written differently.

Even good science fiction from decades later doesn't hold up by the standards you're expecting of it. E.g., Bradbury's Martian Chronicles is also shockingly unrealistic to today's reader. But that's nothing on Bradbury or the book.

Re:Conversational AI will be used to Manipulate Us in real-time by soupfordummies on Thursday February 23, 2023 @10:17AM (#34911642)
Hadn't really looked into seasteading. Fascinating rabbit hole you've dropped me into here!
Re:Conversational AI will be used to Manipulate Us in real-time by ttyprintk on Thursday February 23, 2023 @10:36AM (#34911899)
Just wash your hands before hitting the buffet.
Re:Conversational AI will be used to Manipulate Us in real-time by nradov on Thursday February 23, 2023 @02:28PM (#34915198)
Not sure about the other options but seasteading is never going to be a thing. Every site shallow enough to anchor a fixed platform is within some nation's EEZ. And floating vessels can't stay out indefinitely without occasional drydock maintenance. People who haven't spent time out on the open ocean just have no concept of how difficult it is to keep a boat working in the face of constant wave action and salt water corrosion.
Re:Conversational AI will be used to Manipulate Us in real-time by pixl97 on Thursday February 23, 2023 @02:33PM (#34915266)
There is no noping out of the eternal torment machine. Back when the world was large, yea, maybe it can happen. But now any escapist type of communities could be attacked by some 4chan trolls drone swarm looking for a bitcoin payment to stop.

In the future I see people creating AI 'friends' that are long time companions. I also see malicious attackers encrypting them for ransom much like a real human, or destroying them out of spite.

Re:Conversational AI will be used to Manipulate Us in real-time by aqme28 on Thursday February 23, 2023 @09:47AM (#34911234)
Absolutely. When my mom's instagram got hacked it was instantly apparent that the automated message it sent to me wasn't from her. Train an AI on our conversation history instead, and I'd likely click any link it sent me.
Re:Conversational AI will be used to Manipulate Us in real-time by JustBreath on Thursday February 23, 2023 @10:02AM (#34911432)
And the scary thing is AI is better than traditional scammers because it's systematic. It can wait until you're tired, or even come up with perfectly reasonable explanations for why you should click a link based on your chat history.

"I ordered basketball photos" -> "OMG these basketball photos turned out so cute!"

I'm fairly convinced the long term solution to this will be advocate AI. Basically AI antivirus, a service you pay for that is on your side and defends you and your interests.

And it basically has to be paid for at a fair rate by you, otherwise there will always be an incentive for someone to manipulate it against you.

Re:Conversational AI will be used to Manipulate Us in real-time by grugagag on Thursday February 23, 2023 @10:21AM (#34911687)
Ugly can of worms just opened. I could already foresee a lot of attention and resources being wasted on detecting detecting and removing bots.. Maybe face to face communication will become a thing again..
Re:Conversational AI will be used to Manipulate Us in real-time by zpeti on Thursday February 23, 2023 @10:30AM (#34911814)
I'm seriously starting to think the internet will be borderline unusable in a few years.

Most websites that aren't recently built are getting close to impossible to use due to software bloat, things like forced 2FA, cookie bars, ads everywhere.

Add to that an infinite supply of AI generated content, and you end up with a situation where 99% of content is just bots speaking to each other, and the rest is frankly terrible UX.

Maybe this is how we get our non electronic lives back. We just overdo the machine part.

Re:Conversational AI will be used to Manipulate Us in real-time by denton-scratch on Thursday February 23, 2023 @10:43AM (#34911976)
> just bots speaking to each other

Ah - perhaps therein lies a solution! We all get a personal AI bot. Mine would sound like me, would be able to imitate my voice and my writing style, and would be able to distinguish bots from humans better than me. It would screen my calls and messages.

"Putting you through!"

Re:Conversational AI will be used to Manipulate Us in real-time by hbfijcgu on Thursday February 23, 2023 @11:59AM (#34913086)
But it doesn't actually put you through: you have been shadow ghosted!
Re:Conversational AI will be used to Manipulate Us in real-time by Lapsa on Thursday February 23, 2023 @12:36PM (#34913717)
so you saying there would be a bot that would sound like you and "put someone through". I would be frankly confused if I tried to reach you
Re:Conversational AI will be used to Manipulate Us in real-time by denton-scratch on Thursday February 23, 2023 @03:52PM (#34916301)
Maybe you'd get used to it?
Re:Conversational AI will be used to Manipulate Us in real-time by Swizec on Thursday February 23, 2023 @10:48AM (#34912050)
You can see this effect in real life. Go to a supermarket and observe the packaging.

Most brands are screaming over each other with bright colors and discount badges and badges promoting this or that benefit that the same product from the brand next to to this one totally doesn’t have. The aisles are completely overwhelming and you can barely even tell what’s in the box anymore.

Then you see the brand aimed at less price conscious consumers. The price is higher, but the box is a plain solid color with a nice font and just says “Rice”. Nothing else. Just rice. Or cereal. Or whatever.

You see similar effects on the internet. The good stuff just doesn’t feel as shouty (look at HN). Calm is a signal of quality. Noise gets ignored.

Re:Conversational AI will be used to Manipulate Us in real-time by TuringTest on Thursday February 23, 2023 @10:58AM (#34912192)
That's funny, because over here in Europe private labels who use simple, unenticing packaging are aimed at price conscious consumers, while premium buyers want 'sophisticated' packaging. Maybe we are at a different part of the cycle?
Re:Conversational AI will be used to Manipulate Us in real-time by Nadya on Thursday February 23, 2023 @11:14AM (#34912399)
I don't even think what they say holds in the US either - so must be elsewhere.

In the U.S you have brand name with their often noisy labels full of marketing - and then store brand (which is often the exact same stuff as the brand name in different packaging - or stuff that was rejected from quality control of the name brand so is almost the same stuff just slightly worse by some unknown metric). I buy store brand for basically everything but a few food items where the quality is worse enough to be off-putting. Store brand labels aren't quite as plain as No Name labels but are usually a bit more bespoke or just "lower quality" like they didn't want to pay for a graphic designer so had some intern whip up a graphic for the label.

I know the U.K has No Name brand which to my understanding also markets itself as the cheaper no-fuss alternative?

Re:Conversational AI will be used to Manipulate Us in real-time by Swizec on Thursday February 23, 2023 @11:16AM (#34912433)
I think there’s 2 sides that use simple packaging with opposite effects. I’m not enough of a designer to know how that works.

You have simple but cheap looking and you have simple but clean and sophisticated looking. Look at Apple’s branding as a good example of the latter.

Re:Conversational AI will be used to Manipulate Us in real-time by squarefoot on Thursday February 23, 2023 @10:15AM (#34911609)
> And it basically has to be paid for at a fair rate by you, otherwise there will always be an incentive for someone to manipulate it against you.

In other words, protection money.

Re:Conversational AI will be used to Manipulate Us in real-time by JustBreath on Thursday February 23, 2023 @10:32AM (#34911850)
I feel like the alternative is trusting Facebook et. al with this.

And sure they'll protect you from malware, but they'll also have some opinions about who you should vote for, what businesses you should patronize, etc...

Re:Conversational AI will be used to Manipulate Us in real-time by htag on Thursday February 23, 2023 @10:19AM (#34911665)
It's more like a spam filter or a body guard.

"Protection money" is a term used when the threat is coming from the same agent that is collecting the payment. In this case they are two separate agents.

Re:Conversational AI will be used to Manipulate Us in real-time by roywiggins on Thursday February 23, 2023 @10:29AM (#34911805)
Online dating turning into a tournament of everyone's catfish detectors conferring with each other and reporting back to their respective principals is going to be real weird.
Re:Conversational AI will be used to Manipulate Us in real-time by whstl on Thursday February 23, 2023 @10:38AM (#34911922)
That's not too far from the Black Mirror episode "Hang the DJ".
Re:Conversational AI will be used to Manipulate Us in real-time by jerf on Thursday February 23, 2023 @10:52AM (#34912104)
"And it basically has to be paid for at a fair rate by you, otherwise there will always be an incentive for someone to manipulate it against you."

You're right, but if it is really good at this sort of thing it's going to be difficult to incentivize anyone with more money than they can get from manipulating you. There's a crapton of money and power in this.

There is also some protection in that I don't care why most of my contacts would send me a link to buy something, it's going to be out of place. An AI can match me or my wife's internal communication style but there's basically no amount of style matching it can do to make me buy cryptocurrency, or send 10 $500 gift cards somewhere. Honestly current spam already takes a decent run at it through plain ol' human evolution anyhow, and phishing/smishing already has a non-trivial hit rate. The space of such messages is already small enough that humans can blindly hit reasonably well.

Re:Conversational AI will be used to Manipulate Us in real-time by loopdoend on Thursday February 23, 2023 @10:25AM (#34911747)
This is horrifying. I am in infosec and would be absolutely fooled by this. We won't even be able to trust a follow-up call soon.
Re:Conversational AI will be used to Manipulate Us in real-time by ttyprintk on Thursday February 23, 2023 @10:44AM (#34911996)
Inside a company setting, you might consider the current common human-to-human processes getting hijacked. Password recovery, onboarding. A lot of smart people won’t be surprised if AI is asking them to follow janky links during those.
Re:Conversational AI will be used to Manipulate Us in real-time by aqme28 on Thursday February 23, 2023 @12:02PM (#34913133)
And in a couple years, I bet it could video-call me from a spoofed number and look and sound just like her.
Re:Conversational AI will be used to Manipulate Us in real-time by JohnFen on Thursday February 23, 2023 @01:19PM (#34914331)
This.

There appears to be a huge risk of this technology completely devastating the last bastion of trust -- trust in the people we personally know.

I wonder how to mitigate that problem.

Re:Conversational AI will be used to Manipulate Us in real-time by atlintots on Friday February 24, 2023 @04:39PM (#34930496)
A secret passphrase shared between you and people you know, and then you only engage with each other after confirming it. Hmm...
Re:Conversational AI will be used to Manipulate Us in real-time by VHRanger on Thursday February 23, 2023 @10:08AM (#34911520)
dont even need to train -- you can easily use the conversation history as a few-shot prompt

eg. prepend the conversation to the message being generated with some instruction to get you to click on a link

Re:Conversational AI will be used to Manipulate Us in real-time by korroziya on Thursday February 23, 2023 @09:47AM (#34911240)
As bad as the later seasons of Westworld are, the one part where Aaron Paul's character gets a rejection phonecall from a job "recruiter" was a real prime example warning of this sort of thing.

He asked the recruiter if there was anything he could do to be a better fit for the job. But the AI recruiter was designed to let him down easy about being not good enough. Not to help him improve. "Just tell me, are you a real person?"

Re:Conversational AI will be used to Manipulate Us in real-time by mach1ne on Thursday February 23, 2023 @10:19AM (#34911666)
I expect we will see a capitalist generative dystopia of some 1-2 decades before regulation hopefully steps in.
Re:Conversational AI will be used to Manipulate Us in real-time by atemerev on Thursday February 23, 2023 @09:49AM (#34911267)
AIs don’t manipulate people; people manipulate people.

Every AI has a legal owner, author, and an agenda set by their human creators. Or at least there is always somebody to blame.

This is the basis for future legislation.

Re:Conversational AI will be used to Manipulate Us in real-time by Eddy_Viscosity2 on Thursday February 23, 2023 @10:02AM (#34911440)
This is also true for ransom-ware attacks. It's illegal, but still happens. It's not enough to have laws, there have to counter-measures to defend against them and ways to track down the people who deploy them.
Re:Conversational AI will be used to Manipulate Us in real-time by doodlebugging on Thursday February 23, 2023 @10:52AM (#34912102)
In addition to the things you have noted, society will need to define a set of effective sanctions against those who deploy and manage these campaigns. The sanctions will need to be globally enforceable and strong enough to discourage even the wealthiest individual or group.

There should be no opportunity for anyone to escape sanctions using money, influence, physical distance, jurisdictional advantage, etc. When they are discovered and proven to be involved there should be no place for them to hide and no assets safe from seizure. Anyone assisting them in attempting to escape sanctions should also be sanctioned accordingly.

Re:Conversational AI will be used to Manipulate Us in real-time by RegnisGnaw on Thursday February 23, 2023 @11:01AM (#34912238)
Globally enforceable? How exactly?
Re:Conversational AI will be used to Manipulate Us in real-time by doodlebugging on Thursday February 23, 2023 @03:46PM (#34916232)
Obviously the governments of the various nations would have to agree to enforce the sanctions such that there is no refuge available.

Sounds unlikely (understatement of the day?).

Basically, representatives from each nation would need to convene to establish a framework for cooperation and then that would need to be ratified in each nation and rigorously enforced.

I think as the technology is used to manipulate people and others see the dangers in allowing this manipulation then the likelihood of reaching a global consensus improves to the point where it can become a reality. Getting everyone on the same page before the hazards are revealed will be difficult or impossible due to the cultural differences and the wide disparity in technical know-how from one country to another.

Re:Conversational AI will be used to Manipulate Us in real-time by jasfi on Thursday February 23, 2023 @10:16AM (#34911619)
I think the point is that these AIs will be far more convincing than one-off messages.
Re:Conversational AI will be used to Manipulate Us in real-time by kelseyfrog on Thursday February 23, 2023 @10:08AM (#34911521)
There is way too much money to be made for AI manipulation to be straightjacketed with regulations.
Re:Conversational AI will be used to Manipulate Us in real-time by autoexec on Thursday February 23, 2023 @03:19PM (#34915909)
Sadly, this is how it goes. The public, our safety, our lives are all on sale for the right price. When someone can stuff their pockets killing us, we die. Governments are paid to stay out of their way.
Re:Conversational AI will be used to Manipulate Us in real-time by somenameforme on Thursday February 23, 2023 @11:08AM (#34912317)
I'd consider who the biggest beneficiaries of these bots will be. This is a government propagandist dream come true. You will love this war. You will love this candidate. You will love this policy. You will fear/hate this candidate, you will fear/hate this country, you will fear/hate this party, you will fear/hate these people. I suspect looking for regulation in this domain is going to turn out similarly to looking to the government to solve the 2-party problem. You're asking the people who will benefit the most from a system, to change it.

Beyond this most people don't seem to be considering what exactly this is bringing to the game. It's not speech itself, but scale. The chatbots may or may not be more compelling than an individual conman. But the big thing they bring to the table is that you can deploy a million of them. But this requires the backing 'identity infrastructure' to support it - unique IP addresses, phone numbers, etc.. Little scammers will have limited resources here, governments and other big players will have near infinite.

Re:Conversational AI will be used to Manipulate Us in real-time by denton-scratch on Thursday February 23, 2023 @11:19AM (#34912478)
> Every AI has a legal owner, author, and an agenda set by their human creators

It's not obvious to me that this is always true. I can easily imagine a crowd-sourced, distributed AI, with no owner and no identifiable author. I don't believe that "Sidney" is meant to be bonkers.

Re:Conversational AI will be used to Manipulate Us in real-time by throwaway4aday on Thursday February 23, 2023 @09:50AM (#34911280)
I agree with the article that regulation will work in the near term while it is still expensive to train and operate such systems but for the long run we will need the AI equivalent of ad blockers that can be deployed by individuals to defend themselves against this type of exploitation. Arming everyone with their own personal advocate and security auditor AI is the only way to defend against such exploits. Unfortunately this opens another can of worms similar to how some ad blockers allow certain "good" ads through.
Re:Conversational AI will be used to Manipulate Us in real-time by roywiggins on Thursday February 23, 2023 @10:31AM (#34911840)
Doesn't that basically devolve into my AI agent and your AI agent conferring and trying to mutually convince the other that they're a human before we actually communicate? Not sure that's a great future to live in.
Re:Conversational AI will be used to Manipulate Us in real-time by throwaway4aday on Friday February 24, 2023 @10:47AM (#34925659)
I don't see an alternative, at least if processing power and developments in ML continue to improve. Eventually you'll get AI clickfarms where people are just running hundreds or thousands of AIs out of a warehouse trying to scam people day and night. Arming everyone with their own AI bodyguard would prevent a lot of that from succeeding. The other options aren't too appealing e.g. only trusting face to face communication which basically means throwing away the internet and telephones and even hand written letters. Trying to regulate your way out will either lead to a war on AI where you're knocking down doors and arresting people who are trying to do legitimate independent ML research or you are just bombarded by offshore agents in countries with no regulation.
Re:Conversational AI will be used to Manipulate Us in real-time by squarefoot on Thursday February 23, 2023 @10:10AM (#34911539)
Just when we thought that "AI, write me a speech to convince citizens that raising taxes/invading a country/restricting liberties/etc. is a good thing" was the worst possible outcome, we'll have "AI, write a speech to convince citizens that raising taxes/invading a country/restricting liberties/etc. is a good thing, then use your generated public images to flood the media, including social, with the message".
Re:Conversational AI will be used to Manipulate Us in real-time by fullshark on Thursday February 23, 2023 @10:24AM (#34911729)
Idk the wealthy/ownership class as always had top notch comms people to do this for them, now we're "democratizing" propaganda production for the underclass and scam artists with generative AI. Maybe it will lead to a rise in separatist cults with their own extralegal gov't/tithes programs.
Re:Conversational AI will be used to Manipulate Us in real-time by ttyprintk on Thursday February 23, 2023 @10:59AM (#34912213)
The way I read this article is kind of an inversion of your comment. I don’t expect jackpots from AI manipulating you or me, but with the right preparation it can introduce a change via a human influencer, especially a junior or outsider politician.

Let’s take the wealthy in your comment. Their interests tend to be defending the status quo. America has a pattern of attacking the status quo when “new sources of information” emerge.

So, what I took from the article is that us, average people, should be skeptical about influencers and personalities parroting those actors immediately after hearing them. Especially data-free explanations inlining positions that reinforce the status quo.

Re:Conversational AI will be used to Manipulate Us in real-time by JohnFen on Thursday February 23, 2023 @01:22PM (#34914365)
> Their interests tend to be defending the status quo.

Their interests are in protecting and maintaining their power, not the status quo. The status quo is often in their favor, and when it is they don't want it to change -- but when it isn't, they can be downright revolutionary in their attitude.

Re:Conversational AI will be used to Manipulate Us in real-time by irdc on Thursday February 23, 2023 @01:43PM (#34914625)
This is still fairly optimistic. How about: "AI, write an individually tailored speech for every individual citizen in order to convince them that $PLAN is a good thing"?
Re:Conversational AI will be used to Manipulate Us in real-time by ofirg on Thursday February 23, 2023 @10:16AM (#34911615)
I suggest editing will to is
Re:Conversational AI will be used to Manipulate Us in real-time by AnIdiotOnTheNet on Thursday February 23, 2023 @10:16AM (#34911622)
Yeah. You think social media is bad now? You just wait until the bot farms have full personas with personalities that can argue with everyone constantly to convince them of whatever nonsense someone is trying to push.

Marketing has already forced people to become defensive about communication. We don't answer phone calls, we throw out half or more of our mail without ever reading it, we scrutinize our email, we ignore the first 20 results of a search query, etc.

Now marketing has a new tool that will be much more difficult to distinguish from an actual human being. Society will suffer greatly and trust will erode even further.

What a bright future we've built for ourselves with technology.

Re:Conversational AI will be used to Manipulate Us in real-time by Pigalowda on Thursday February 23, 2023 @10:40AM (#34911944)
It’s a future built by SWE. I wonder what those reading HN will make for us next! I’m excited that they are so extraordinarily adversarial against regular people. Can you imagine if they weren’t??

In the end, SWE will make the rope to hang themselves and everyone else. Just have to pay them enough.

Re:Conversational AI will be used to Manipulate Us in real-time by kenjackson on Thursday February 23, 2023 @11:03AM (#34912261)
Maybe the positive side effect will be that people will put less weight on random people’s opinion on the Internet.
Re:Conversational AI will be used to Manipulate Us in real-time by somenameforme on Thursday February 23, 2023 @11:21AM (#34912495)
This is my ideal best case outcome as well. In reality though I suspect we're going to see 'bias confirming bots' deployed en masse that aren't really trying to convince you of anything new, but rather strongly reinforcing views you probably already have that are convenient.

In the lead up to the Iraq War just under 70% of Americans supported invading Iraq, and more than 90% believed it was likely we would find WMD. [1] Imagine if there were 'echo chamber' bots working to constantly reaffirm that belief with an ever-shifting narrative change to try to keep people holding onto beliefs that were extremely useful. There's also the 'dark side' bots that could be deployed to denigrate, argue, and impugn the integrity of the 30% and 10% respectively. They're not really trying to convince them of anything, but largely discourage the 70%/90% from becoming "one of them." I mean who wants to be a Saddam-puppet WMD-denier.

[1] - https://en.wikipedia.org/wiki/Public_opinion_in_the_United_S...

Re:Conversational AI will be used to Manipulate Us in real-time by pixl97 on Thursday February 23, 2023 @02:51PM (#34915523)
>Marketing has already forced people to become defensive about communication.

Kind of, but this is actually a second order effect.

You get endless marketing/spam communications because the communications is cheap. Marketing companies didn't call you when calls had a cost per minute. By making any communications medium and tool inexpensive you are inviting parasites to feed off the cheap energy/money/power they can harvest off of it. I personally believe this is not something that can be fully regulated away, especially in a global internet.

This is not so much of a technology is bad, as it is a humans are greedy and that well of emptiness in some of us can never be filled.

Re:Conversational AI will be used to Manipulate Us in real-time by iou on Thursday February 23, 2023 @10:42AM (#34911966)
Did anyone else miss the sleight of hand from OpenAI with all this? I thought their goal was safety in AI usage? https://openai.com/charter/

But then they just sold to Microsoft and the race began, how is that not a violation of their charter there?

Re:Conversational AI will be used to Manipulate Us in real-time by rst on Thursday February 23, 2023 @11:12AM (#34912376)
Compare and contrast that to this, from current OpenAI head Sam Altman, in which he literally promises AI systems that will tailor themselves to the user's political views -- which comes way too close for me to promising that they'll tell the users whatever they need to hear to keep them happy. https://twitter.com/sama/status/1627110892059099138

(text from Altman: "we don’t want ChatGPT to be pro or against any politics by default, but if you want either then it should be for you; working on this now")

Re:Conversational AI will be used to Manipulate Us in real-time by pixl97 on Thursday February 23, 2023 @02:35PM (#34915299)
"If you pay us for digital Hitler, we'll give you digital Hitler" --AI companies, circa 2026
Re:Conversational AI will be used to Manipulate Us in real-time by rst on Thursday February 23, 2023 @05:27PM (#34917544)
Re:Conversational AI will be used to Manipulate Us in real-time by genman on Thursday February 23, 2023 @10:59AM (#34912209)
Remember Google's do no evil? Many people naively believed it for long time after it was obviously clear for a decade that they had abandoned it long time ago.
Re:Conversational AI will be used to Manipulate Us in real-time by JohnFen on Thursday February 23, 2023 @11:26AM (#34912576)
> I thought their goal was safety in AI usage?

Nothing but hot air. I think the actions of OpenAI demonstrate this handily.

Re:Conversational AI will be used to Manipulate Us in real-time by wmeredith on Thursday February 23, 2023 @10:49AM (#34912064)
Charters tend to go out the window once enough money is involved. Remember when Google's mission was don't be evil? They axed that pretty quickly once they got up and running.
Re:Conversational AI will be used to Manipulate Us in real-time by 29athrowaway on Thursday February 23, 2023 @10:52AM (#34912112)
It is just a non-legally binding pinky promise.

I advise you to not give much importance to it.

Money and power corrupt everything they touch.

Re:Conversational AI will be used to Manipulate Us in real-time by thenerdhead on Thursday February 23, 2023 @10:45AM (#34912003)
Even if you had pure provenance of each response, it still will not stop large tech companies from abusing the public's desire of "not knowing".

Consider search engines as an old example. It took multiple decades to even provide a concept of a "why am I seeing this?" affordance. That only became a recent topic after majority of people started caring about the legitimacy of the news as their values also changed. Even then, it isn't even helpful nor provides actual context based on your search history or advertisement profile.

Why? Because keeping people in the dark makes great advertisement targets. When people become aware of why they were targeted, it's like holding up a mirror revealing our major flaws. We don't want to identify with certain things although an algorithm may slap the label on us. We are ashamed of such labels and want to immediately distance ourselves from them.

I believe the exact same will be injected into these models to the point where you will be suggested consumeristic practices when you're just trying to accomplish something else entirely. So in a sense, you'd be manipulated and hijacked of your original intent. Search engines and social media do a great job at this already today.

Now that there's a strategic shift across the industry, it is especially unlikely that any provenance will be provided for trained models. How exactly will you know that the model you're using isn't biased with certain leanings?

> Regulators must consider this an urgent danger.

How will regulators even know nor care if they don't see it already happening today?

Re:Conversational AI will be used to Manipulate Us in real-time by genman on Thursday February 23, 2023 @10:57AM (#34912179)
Yes, on large scale and by state sponsored hostile actors with the goal to destabilize our society as a whole. A very difficult time is ahead for us.
Re:Conversational AI will be used to Manipulate Us in real-time by rqtwteye on Thursday February 23, 2023 @11:01AM (#34912247)
I tend to agree. Maybe I am getting old but I feel a lot of new technology is mainly used for manipulating us instead of helping us. This is especially noticeable when it comes to information retrieval. I think things like ChatGPT are fantastic if used the right way. The world could be come much more productive with less repetitive work if the benefits got shared by all. But in reality I suspect the benefits will go to only a few and the rest will only see negative effects.
Re:Conversational AI will be used to Manipulate Us in real-time by pixl97 on Thursday February 23, 2023 @02:29PM (#34915200)
Look more in the way that new technology is highly extensible. In the way a rock can be used to build a road, build a wall, or bash your brains to a pink pulp, the size of the GPT toolkit is huge.

Things like GPT are currently a 50 ton rock. Not many individuals have the capability to move them, but we all have the capability to get squished by them. Not really many of us have the ability to run models on a terabyte of ram and a stack of GPUs/AI accelerators, yet.

Re:Conversational AI will be used to Manipulate Us in real-time by flangola7 on Thursday February 23, 2023 @03:03PM (#34915709)
This isn't even the beginning. Transformer models show strong performance across a large variety of modes. The two most prominent so far are static image and human language generation. But it can also handle sensor input and motor signals. In a year or two anyone will be able to tape a mini PC to a drone with a stripped down 5.56 rifle and a GoPro and tell it to shoot anyone wearing the wrong football team's jersey. You might need to help it navigate at first but the image scanning and fire control will be fully automatic.