"Hey mom, it's X. Sorry to leave this voice mail, I got in a bit of a car accident and lost my phone. I need to get back home, could you send $$$$ to ####?"

Just have to establish real life security questions.

They did this in one of the Harry Potter books (the sixth one?) since it is easy for them to impersonate other people.

If you can’t tell me my favorite flavor of jam then you aren’t getting my money.

What is your favorite flavor of jam?

This wouldn't work for me. I always pretend to like something and then change my mind the next day to piss people off. Maybe just a call to ensure it's legit. Vocal deepfakes will never mimic voices and ways of talking in real time, it will always sound "off". It doesn't make it any less dystopian, though.

Are you my wife?

"Your foster parents are dead"

Eh, if they hack any adtech provider they may be able glean that you like loganberry.

I'm sure it's going to get exponentially better, but right now ChatGPT sounds like one very vanilla guy. The other day I asked it to write me a conversation between David Foster Wallace and Ernest Hemingway, and it came out something like (paraphrasing):

HEMINGWAY: I notice you write about political issues and philosophy.

WALLACE: Thank you, I noticed you write about raw experience and manliness.

HEMINGWAY: Well, whatever our differences I feel like we can get along and work together. We will soon be fast friends.

I tried to get it to intentionally respond with grammatically incorrect sentences and it had a lot of trouble. I basically couldn't get it to work.

Has ChatGPT actually read any books (apart from snippets on the Internet), or has that been avoided for copyright reasons?

I wonder how long until someone pirates every ebook on Z-Library or SciHub and trains a language model off that. Imagine a language model that's read every book and every scientific paper.

I wonder if that would even generate good outcomes given how much of the data in some disciplines is conflicting, and if you don’t have a good way to get it to ignore bad subsets of that data it could be more likely to hallucinate convincing scientific sounding explanations based on a preponderance of bad poorly researched evidence.

There’s a lot of papers that discredit older papers or work or entire subfields of study and the current large language models would just all all of it and go “these are words” with no analytical reasoning applied.

These are predictive mechanisms not analytical ones and until we get a better handle on how we can make them “stop and think”… I’m just not sure how much benefit each larger dataset will add beyond “sounds more human” while it’s core problems of “still hallucinates” and “can’t do basic reasoning” remain.

Also one question I’ve got on this front is how much duplicate data is processed out of the input sets? Using LibGen as an example there’s a lot of books that get reprinted and uploaded multiple times in different formats… does having these remain in the data provide a “valuable bias” or is it something that needs removing? Do the people building large language models pre-process their data to de-duplicate out these kinds of things from their current data sources?

I asked it to make a certain stand up bit in the style of various comedians. The Seinfeld one was reasonably good but the Chris Rock one was off because it wouldn’t curse.

You can already fool real world voice authentication systems. No need to phish for credentials when the voice is the credential: https://www.theguardian.com/technology/2023/mar/16/voice-sys...

It's kind of amazing we're not seeing more of this already. It's a trivial application of the tech. We've already achieved a culture where the more batty something is the more likely people are to believe it, and now we have an infinite, personalized quackery generator.

Without a massive cultural revolution putting the brakes on this, which I can't imagine coming from anywhere, I fear we're barreling into a future where nothing is true anymore, a hall of mirrors where the capacity to fake anything at all far surpasses anyone's ability to make sense of it.

MacLuhan and others have already talked about this, from way back in the radio and TV generation.

I don't say this to mean, "oh, and they were wrong because everything turned out OK" If anything, we have had generations that grew up with this dysfunction in a normalized way, and it took another advanced in tech to really see it again.

(To take it further back into history, I think I now get what Cervantes was trying to say with the image of Don Quxiote tilting at windmills. It seemed so absurd, amusing, and quaint when I read it as a teen).

If anything TV and radio presented a larger shared reality, even if it was one filled with falsehoods. What is culture but a shared set of values?

What we are growing now is a system that can give each and every one of us our own reality, and if it succeeds, what of society?

There are some fascinating areas to explore those questions from the perspective of shamans, mystics, and yogis.

We can just make the AI phishing bots talk to a Lenny 2.0 trained to tie them up in endless conversations. AI interacting with AI will be the next economic boom fomenting a wave of new wealth creation.

Would it really be wise to run a criminal operation while piping all your requests through Microsoft servers, considering the history of the company collaborating with law enforcement?

It is trivial to get access to some else’s router & credit cards in 2023.

Years and years of bad software practices, corporate cost cutting measures resulted in a planet scale cybersecurity mess.

Current hacking scene is more similar to 80s than early 2000s, it is extremely hard to catch advanced actors.

TikTok lady AI voice starts talking, hangs up phone

Imagine an AI voice of Trump that phones lonely old people to thank them for coming to his rally. After 20 minutes of small talk he asks them to donate to his campaign via bitcoin...

Even simpler. Have the voice give them a short onboarding call, but then have it regularly call them again using reverse charging, and then chat with them. Convince them that they've formed a friendship.

Lonely old people sophisticated enough to do bitcoin?

Color me skeptical.

They get conned into sending money from Bitcoin ATMs[0]

[0] https://www.atmmarketplace.com/news/elderly-michigan-couple-...

I'd imagine an AI could quite patiently walk them through how to set it up and link their bank accounts. Honestly, it could be really helpful for tech support.

I know at least a couple. Doesn't take much sophistication with all the hype the last few years and different apps making it easy. It's essentially as easy as using any other mobile payment tech e.g. mpesa or alipay