If an AI fake-porn of some ordinary person involving a minor was unleashed, think of the utter shame and horror they would be treated by people for the rest of their lives, even if it were proven false.

No one would believe them, work with them, hire them, rent them, they would wish they had been lynched instead of the life they live.

It happens once, then some wag creates videos of all the mob leaders blaspheming or whatever. Undermines the idea that the videos are the root cause.

Also there are no mob 'leaders'. These are everyday folks, these are you and me and my neighbours.

A student pissed at a teacher could create a fake AI video of his teacher at night, and wake up to no pesky teacher and feel no remorse over what happened while he slept.

The student was not a mob leader, just some dumbass who was angry at their teacher for giving bad grades, and had access to simple AI tools (soon available online for a trivial fee, if not free).

____

Be honest, are you 100% sure of the status of that facebook account you deleted in 2018? Zuck never deleted it, that content is still there to be mined and abused. Also not that hard to just log in to the damn thing and post something spicy.

I don't remember if old posts can be edited in FB, cause that would allow the person to really gild the lily.

I believe you are right. When the first unbelievable clips appeared on Facebook (where the most gullible people are), you could see the old ladies cheering "oh how wonderful! where did you film that?" Now in the comments section you will mostly see people saying "sheesh, stop polluting my feed with these fake AI-generated spam". Don't get me wrong, many people still fall for them, but the tendency seems clear - after all, we managed to survive because of learning.

Remember the post is about cloning a youtuber's voice using AI; most people were thinking of the copywrite aspect of it, but I wanted to share with people what my fear was.

My fears are, things are already bad w.r.t lynch mobs before AI, after AI things will only get worse with fake but realistic sounding voice notes etc. That's my fear, doesn't have to be yours.

You don't have lynch mobs (hopefully) but surely you can think of other problems that a fake AI image/audio/video can cause, and just low the barrier is, and just how good the tools are.

It’s fundamentally a societal issue, not a technological one. Yes, it’s scary what effect AI could have on lynch mobs, because lynch mobs are scary.

The only way forward is making sure most people know that digital content can be easily faked. If you hand out good faker tools for free, it will happen faster

Remember when people believed that camera took our soul when we are caotured on image?

Or https://www.npr.org/2024/09/19/nx-s1-5114047/springfield-ohi... , where repeating racial libel causes a public safety problem.

While this kind of incitement in no way requires AI, it's certainly something that's easier to do when you can fake evidence. See also https://www.bbc.co.uk/news/articles/c5y87l6rx5wo

In the US, freedom of speech means the government won't stifle you, but the court of public opinion will crucify you for saying the wrong thing. In the linked case, a school principal was framed for being a racist, a fire able offense. Better than being killed by religious extremists, I suppose, but still not great. Thankfully in this case they were able to find the perpetrator who faked evidence, but we have a problem.

The solution is not to ban technology (it's impossible), but to create a more decentralised society where the majority of voters, which is easily brainwashed, doesn't get to dictate the life of the minority. We need to destroy and dismantle all the gigantic entities that ruin our life for the advantage of the few, whether they are the government or corporations ruled by billionaires controlling the government.

We also need to create a better society with less mental illness, where people are not depressed by how unfair life is that they go on and kill kids in school - but I believe if we reduce large societal inequality this will sort itself out over time.

Parents in my circle are already banning social medias for their kids until they're adults - so over time we'll evolve to get better at ignoring all the crap we get exposed to.

That take pretty much throws the shooters' preceding years of anguish under the bus to make a political score. "Ban the wrongspeech!" is a seductively easier proposition than ending youth bullying and ostracism.

> youth bullying and ostracism

... which itself counts as free speech?

A lot of people get bullied in a lot of countries. Few of them turn to mass murder. The mass shooter phenomenon is mostly America, plus a few people who've been radicalized by the same racist discourse on the same message boards (Anders Brevik, NZ mosque shooter).

In quite a lot of cases closer investigation suggests that they were ostracized precisely because they'd already chosen hate.

So yeah, I would call this "wrongspeech" and wrongspeech is already banned.

Sure, distrust everything digital, but what if only evidence of someone doing something wrong is digital?

With celeberaties, atleast they are famous enough that the idea of fakes exist... but you are Johnny nobody, that's not a fake, you just got caught and don't want to admit you said/did it!

The court of public opinion is much smaller and personal, and 'evidence' is much more realistic and detailed.

Nothing new.

The trick is being rich without being famous!

It’s like if someone said “I’m scared of someone bringing a semi-automatic weapon to my school and doing a mass shooting. My country has lax laws about guns and their proper use”. And then you said “in your case the problem is not guns, it’s your country”.

I mean, it’s technically true, but also unhelpful. Such ingrained laws are hard to change and you can be placed in danger for even trying.

Before someone decries the gun example as not being comparable, it is possible to live in a country with a monumental number of guns and not have mass murdering every day. It’s called Switzerland.

But let’s please stick to the subject of AI, which is what the thread is about. The gun example is the first analogy which came to mind, and analogies are never perfect, so it’s unproductive to nitpick the example. I don’t mean to shift the conversation from one contentious topic to another.

The US has both problems: widespread availability of weapons and a high level of freedom to incite violence through spreading lies about groups. Which is why it sees much more of these incidents than countries which have similar levels of gun ownership.

The non-gun version of the problem is mass stabbings, which are less lethal.

Vs the US, where there are loopholes that you can use to avoid even a basic background check, and then use it for the very first time to shoot someone.

Second, you learn a certain respect for the firearm and are expected to observe strict safety rules when handling it. That gives you a kind of psychological flinch when you consider doing anything out of the norm with it.

The difference might be in what guns mean to Swiss people vs. the US.

A gun in the US has this "if the government becomes destructive of these ends we can start shooting" connotation. A gun is there for self defense. Someone doesn't get off your law, you use your gun.

In Switzerland the gun and the compulsory military service you mention is there for the people to protect their country and fellow countrymen. You are trained in defending your neighbor, who just stepped on your lawn against outside aggressors.

In the UK a government was just elected with a historical absolute majority by only ten million people, and now first time offenders are being sent to prison for making stupid offensive statements online.

> In a now deleted post on her X account, Mrs Connolly, from Northampton, wrote: “Mass deportation now, set fire to all the f*** hotels full of the bastards for all I care… If that makes me racist, so be it.

> The court heard Kay copied, pasted and uploaded Mrs Connolly’s post at 12.27pm on Wednesday from a BBC News report and added the hashtags #standwithlucyconnolly #lucyconnolly #f**northamptonshirepolice #conservative #FaragesRiots #RiotsUK and #Northampton.

> As well as a post which urged people to “mask up” during a protest targeting an immigration law firm, Kay tweeted to his 127 followers at 2.34am: “That’s 100% the plan, plus gloves. No car either so no number plate to trace and a change of clothes ready nearby.”

https://www.independent.co.uk/news/uk/crime/northampton-bbc-...

My point was that AI faked content could certainly get you locked up in the UK.

As far as I can tell the collective conscious of every country is swayed by propaganda.

A written headline is enough to incite rage in any country much less a voice or video indistinguishable from the real thing.

Folks in “developed’ countries have their lives destroyed or ended all the time based on rumors of something said or done.