News for hackers. Stuff that matters.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by jacquesm on Monday January 09, 2023 @07:24AM (#34309545)
"Hi mom, it's Chrissy, I am calling you from someone else's phone, my phone got stolen. Do you have a pen handy, I'll wait. Ok, here is my new number, take the old one off your contacts right away, surely some scammer has it, and can you forward me 1000 bucks or so for a new phone, I'll pay it back when I have the new phone set up, here is my account number...."

When it's a text message it's likely going to ring a bell somewhere (hopefully an alarm bell), if it is a voice message and it sounds like the original many more people are going to fall for it.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by sendfoods on Monday January 09, 2023 @07:33AM (#34309617)
My god...I haven't really thought about scamming at all, I was "just" worried about manipulation of media and politics. New nightmare unlocked.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by hackerman123469 on Monday January 09, 2023 @08:12AM (#34309933)
"Okay, let me just video call you because I also need to show you something" and then the scammer hung up and was to never be seen again.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by probably_wrong on Monday January 09, 2023 @08:26AM (#34310033)
FYI, there's the following trick I read here some time ago:

The scammer starts a video call with the person they want to impersonate and record it. When that person takes the call, the scammer doesn't say anything. This creates a 5-10 seconds video of the person looking at the camera waiting for the scammer to say something, until they get fed up and hung up.

The scammer then calls the victim. They offer a video call as verification straight away, and they play that short video.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by Loughla on Monday January 09, 2023 @09:13AM (#34310504)
How is a 5-10 second video of someone either not saying anything, or saying, "Hello, who is it?" over and over convincing at all?
Re:Microsoft’s new text-to-speech model can duplicate anyone... by wccrawford on Monday January 09, 2023 @09:29AM (#34310683)
I imagine they say something like, "I couldn't see or here you. Did you see me?" through the other channel, and because modern tech messes up enough, it'll be believable to a lot of people. Especially the most gullible.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by citizenpaul on Monday January 09, 2023 @10:50AM (#34311650)
HN gets so caught up in logic. Has no one watched interviews of scammed people? Huge numbers of them report that they were suspicious it was a scam but they were so worried for a loved one they sent the money anyway because it was worth the risk to them just in case. Anything that helps push scams in the even slightly plailusible direction will make scams more successful.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by pictureofabear on Monday January 09, 2023 @08:27AM (#34310042)
All of these advances in technology have made the internet much less trustworthy. I wonder if we'll eventually hit a threshold where nothing is done virtually anymore because you can't trust it.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by sofixa on Monday January 09, 2023 @09:03AM (#34310404)
I think the work being done in the opposite direction is going to make it so the Internet can be used for high trust actions too.

E.g. in the EU we have ID cards and passports with biometric information and NFC, and there was a beta of a phone app by the French government where it would read your photo from the ID card/passport, and compare with a video selfie. That way you get fully local and secure identification allowing you to do stuff online that would otherwise require you to show up in person to a government office.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by jacquesm on Monday January 09, 2023 @10:34AM (#34311442)
That 'video selfie' is untrustworthy, which was sort of the point.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by sofixa on Monday January 09, 2023 @10:40AM (#34311514)
It's taken by the app directly, so it's somewhat hard to pass fake 3D video input to it.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by int_19h on Monday January 09, 2023 @04:56PM (#34316759)
Only if hardware is completely locked down.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by jacquesm on Monday January 09, 2023 @05:01PM (#34316828)
Scammers are well incentivized to find bypasses and feeding an alternative video stream into an app on a device that you control physically isn't all that hard for a determined person. I can think of three ways to do this right off the bat and if I think about it a little longer I'll probably find some more. I sincerely hope that that is not the last line of defense.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by jvm___ on Thursday January 12, 2023 @09:57PM (#34363441)
The open internet will be a dark forest, beautiful, but populated by metal-brains. Meat-brains will retreat inside trusted spaces below or above the empty forest.

https://maggieappleton.com/ai-dark-forest

Re:Microsoft’s new text-to-speech model can duplicate anyone... by teekert on Monday January 09, 2023 @07:30AM (#34309589)
So, we need to work on this ringing bells. Sure some people will fall for it initially, but then we adapt.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by Gigachad on Monday January 09, 2023 @07:39AM (#34309664)
Scammers have been scamming since the beginning of society
Re:Microsoft’s new text-to-speech model can duplicate anyone... by hef19898 on Monday January 09, 2023 @07:58AM (#34309811)
Oh, great! Now we have to come up with proper code words and code books for everyday life! I', not worried so, some AI solution will help us with that for sure.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by bambax on Monday January 09, 2023 @08:01AM (#34309845)
You can still

1. Ignore voice messages

2. Ask pointed questions to any live caller, esp. one calling from a new number and asking for money...

Re:Microsoft’s new text-to-speech model can duplicate anyone... by counttheforks on Monday January 09, 2023 @08:39AM (#34310151)
So now scammers will have to first scrape your & your families social media, then run that through ChatGPT to answer any questions.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by nullsense on Monday January 09, 2023 @09:07AM (#34310458)
Let the games begin.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by DonHopkins on Monday January 09, 2023 @10:30AM (#34311388)
The one earnest question that will ALWAYS make scammers hang up immediately:

"In case we get cut off, what phone number can I call you back on?"

If they actually give you a number instead of hanging up (which has NEVER happened in the many times I've used this line), just hang up immediately and call them back, to see if they answer.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by 4ggr0 on Monday January 09, 2023 @08:18AM (#34309973)
"Hey mom, could you tell me the name of your first pet, your favorite song and your birthday? Also, please fill out the captcha from this URL. xoxo chrissie"

EDIT: Also looking forward to AIs scamming each other, fully-automated.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by chirau on Monday January 09, 2023 @09:13AM (#34310509)
But AIs can't own crypto... or can they?
Re:Microsoft’s new text-to-speech model can duplicate anyone... by mikro2nd on Monday January 09, 2023 @09:55AM (#34310989)
What's to stop them? A wallet is just a keypair.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by 4ggr0 on Monday January 09, 2023 @10:05AM (#34311100)
Maybe they can bootstrap their own crypto? Or just open Monero wallets. If we're at the point where they can imitate voices and human behaviour, they might even pass KYC checks after stealing someones information.

As soon as AI gets out of hand and starts randomly hacking things we're probably out of luck anyways.

“If the rise of an all-powerful artificial intelligence is inevitable, well it stands to reason that when they take power, our digital overlords will punish those of us who did not help them get there. [...]” - Bertram Gilfoyle

Re:Microsoft’s new text-to-speech model can duplicate anyone... by pjc50 on Monday January 09, 2023 @12:02PM (#34312603)
I don't see what relevance that has other than making it easier to scam people irreversably?
Re:Microsoft’s new text-to-speech model can duplicate anyone... by wolpoli on Monday January 09, 2023 @05:48PM (#34317567)
They just need to earn their own crypto somehow. The AI could do then quite a bit even though they do not have Personhood, which could include lobbying politicians for Personhood.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by aceazzameen on Monday January 09, 2023 @08:47AM (#34310232)
This and who knows what else as time goes on. It might be a good idea to not talk on phones anymore unless you already know the contact.

In this bleak scrape the bottom of the barrel world, I can't help but think it's only a matter of time before customer service departments start selling voice data to the highest bidder. And to counter that there will be services where everyone ends up using voice changers, except close personal contacts. I want this fiction to stay fiction.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by MonkeyMalarky on Monday January 09, 2023 @09:26AM (#34310652)
>It might be a good idea to not talk on phones anymore unless you already know the contact.

As a millennial, I'm way ahead of you on that one. The government and telecom companies just need to crack down on number spoofing. Any day now..

Re:Microsoft’s new text-to-speech model can duplicate anyone... by 1over137 on Monday January 09, 2023 @09:08AM (#34310459)
>It might be a good idea to not talk on phones anymore unless you already know the contact.

As opposed to what? Writing is even easier to forge, no?

Re:Microsoft’s new text-to-speech model can duplicate anyone... by MisterTea on Monday January 09, 2023 @09:02AM (#34310397)
My grandmother was called by a scammer claiming they were me, said they were arrested and needed bail money or some nonsense. She knew it wasn't my voice, asked two questions which were dodged then told them where to go and how to get there. It left her very shaken and my mother had me call her to assure her I was fine and as soon as she heard my voice she bellowed "Now that's my grandsons voice!" With this tech all they need is a few seconds of my voice.

With AI we have lost all privacy and identity. This makes nonsense like novelty theory and timewave zero seem less nonsensical. Lets hope we make it through the great filter.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by nullsense on Monday January 09, 2023 @09:12AM (#34310501)
>Lets hope we make it through the great filter.

My thoughts on this are that oftentimes things at the extreme ends of the distribution are usually in some way, shape or form problematic. Compared to all other living creatures we know of, humans are right at the extreme tail end of the intelligence distribution, and the result is somewhat pathological when compared to other creatures. I think nothing that's north of a certain threshold required to make advanced technology can make it through the great filter without destroying itself in the process.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by galangalalgol on Monday January 09, 2023 @10:34AM (#34311439)
My thoughts are that getting noticed by another culture is the great filter, so cultures make it through occasionally, but we will not notice them. "The benefits of not being seen"

Of all my concerns for humanity, excessive intelligence is not one of them...

Our earliest (and easiest to detect) transmissions should already be far enough away to be far below ambient noise I think/hope. So maybe we made it through unless someone starts shouting. Own-goals are also certainly a thing, but it would take a serious mistake to end all of us, or even set us back that far technologically on a geologic time scale.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by nullsense on Monday January 09, 2023 @10:45AM (#34311580)
>Own-goals are also certainly a thing, but it would take a serious mistake to end all of us, or even set us back that far technologically on a geologic time scale.

I must be a pessimist in this regard. I don't necessarily think it would take a serious mistake, rather just the nature of complexity itself will likely become a factor at some point and I can imagine certain scenarios playing out based off that.

When you compare the length of time of humanity spent pre-neolithic-revolution with the short ~14,000 odd years since, I can't help but feel our current trajectory won't see us reach hundreds of thousands more years into the future without a serious backslide in that time.

Humanity is pretty darn resilient, but the last few years have highlighted to me that modern day life is some weird fantasy land we've created for ourselves that has an expiry date at some point in the future.

Re:Microsoft’s new text-to-speech model can duplicate anyone... by bmacho on Monday January 09, 2023 @09:10AM (#34310482)
"Hi Chrissy, can you touch your ID to the phone so I can know that it is you" call ended

The government just should give you a card sized mini computer with 2 large primes stored in it, and everyone is perfectly fine. In fact, more secure, than the current situation?

Re:Microsoft’s new text-to-speech model can duplicate anyone... by addandsubtract on Monday January 09, 2023 @09:31AM (#34310705)
"My ID got stolen with my phone..."

It'd be better to ask something personal. "What did we do to Christmas/Birthday/[other event] last year? Who was with us?"

Re:Microsoft’s new text-to-speech model can duplicate anyone... by lm28469 on Monday January 09, 2023 @09:30AM (#34310686)
In the US where stealing a SSN is enough to impersonate someone I doubt the government would provide anything to mitigate these
Re:Microsoft’s new text-to-speech model can duplicate anyone... by galangalalgol on Monday January 09, 2023 @10:21AM (#34311268)
And telephone numbers can be spoofed for voice or text in the US. Not sure when the problem will get big enough that signed comms become normal. I guess imessage probably is? But only between users of apple devices.
Re:Microsoft’s new text-to-speech model can duplicate anyone... by DonHopkins on Monday January 09, 2023 @10:28AM (#34311364)
Two large Prime minicomputers sounds pretty heavy for people to carry around with them all day!

https://en.wikipedia.org/wiki/Prime_Computer

https://en.wikipedia.org/wiki/Prime_Computer#/media/File:Pri...

Re:Microsoft’s new text-to-speech model can duplicate anyone... by mensetmanusman on Monday January 09, 2023 @09:15AM (#34310528)
Society will respond by having secret keywords that you know from a young age in your family to verify or to point to an emergency that is ongoing.

“ Here’s my new account number. It’s really me. Slippers in the toaster.”