News for hackers. Stuff that matters.
Re:Jeff Geerling – They stole my voice with AI by godelski on Sunday September 22, 2024 @04:43AM (#41615622)

  > what if AI was used to imitate a person saying something blasphemeous?
I've been contemplating writing an open letter to Dang to nuke my account. Because at this time you can likely deanonymize any user with a fair amount of comments. As long as you can correlate. You can certainly steal their language, even if not 100% accurate. It may be caution, but it isn't certain that we won't enter a dark forest and there's reason to believe we could be headed that way. But at the same time, is not retreating to the shadows giving up?
Re:Jeff Geerling – They stole my voice with AI by ryzvonusef on Sunday September 22, 2024 @05:06AM (#41615715)
The problem I fear is that, let's say you once had a facebook account, we all deactivated our accounts when there was wave against Zuck a few years back, but as we know, facebook doesn't really delete your account.

Now imagine that account was linked to a SIM. It's trivial for a nefarious actor to get it re-activated, infact there was a video by Veritasium just today where they didn't even need your SIM.

But even if they are not that hi-tech, it's not that hard to get a SIM issued in your name, or other hacks of a similar nature, we have all heard of stories.

Worse, you lost that SIM a decade back, the number gets back into the queue, and is eventually re-issued to someone new... and they try to create a facebook account, and are presented with yours.

They can then re-activate your old facebook account, and post a video/audio/text of "godelski" saying they like pineapple on pizza. and before you can defend yourself, the pizzarias have lynched you.

(I dare not use a real example even as a jest, I live here)

Are you 100% sure of all your old social media accounts, all the SIM you have ever used to log-in to accounts?

We leave a long trail.

Re:Jeff Geerling – They stole my voice with AI by rustcleaner on Sunday September 22, 2024 @01:38PM (#41618573)
I think we need a data Great Reset: all data (and I mean all) must be deleted by a certain date in the future, and all newly collected data must have provenance so in the future the customer can find out that Ford got information from Databroker A which bought it from B which bought it from Microsoft which collected it from what you type in Windows. Companies holding activity data without provenance get the crack-cocaine dealer treatment: officers in jail, everybody fired, investors and lenders fucked, company liquidated and forfeited.

While it is impossible to take back a disclosed secret, we can create a legal framework which issues immediate business-stopping corporate death sentences for spreading that data without consent (or after you revoke consent, yes it needs to be revokable). Your data is so valuable, because now that we have universal function approximators you can be simulated (to a prediction horizon) and that simulation interrogated.

Re:Jeff Geerling – They stole my voice with AI by danieldk on Sunday September 22, 2024 @04:49AM (#41615645)
There should be a way to cryptographically sign posts (everywhere). I know, building a web of trust sucks, etc. But if there was someone with your username signing with a particular key for 10 years and then suddenly there is something controversial somewhere with a different key, something fishy is going on.

Of course, this could be misused to post something with plausible deniability, but if you want to say something controversial, why wouldn't you make another account for that anyway?

I know that one could theoretically sign posts with GPG, but it would be much nicer and less noisy if sites would have UI to show something like: Signed by <fingerprint>, key used for N years.

One issues is that most social media want your identity to be the account on their service and not some identity (i.e. key) that you control.

Re:Jeff Geerling – They stole my voice with AI by godelski on Sunday September 22, 2024 @04:59AM (#41615669)

  > There should be a way to cryptographically sign posts (everywhere).
This just confirms it is me. Which yes, reduces the problem of me being replicated, but does not do anything for my anonymity. That part may not seem important to you, considering you use your real name, but it is to me. It allows me to be more open.

Key signing will not be the solution because it isn't addressing the problem. It exacerbates it.

Re:Jeff Geerling – They stole my voice with AI by danieldk on Sunday September 22, 2024 @05:03AM (#41615698)
This just confirms it is me. Which yes, reduces the problem of me being replicated, but does not do anything for my anonymity. That part may not seem important to you, considering you use your real name, but it is to me. It allows me to be more open.

Ah, think I see your point. Your worry is that language use, etc. could be use to deanonymize you, by correlating with text that was not written anonymously. But that's a separate issue from voice or writing style cloning to pretend it's you that said it. In the latter case you could use a pseudonymous signing key?

I agree that deanonymization is an issue that is hard to tackle. I wonder if someone studied how unique writing style is. E.g. browser fingerprints are fairly unique, but I wonder to what extend you can filter a person from, say a pool of 100 million, using writing style alone (grammar, vocabulary use, etc.). I guess it becomes quite easy if you engage in a lot of domain-specific discussions and use their vocabularies.

E.g. if I'd talk about Marlin-kernels here, you could probably narrow me down to a few hundred people. Throw in another comment about the Glove80. Maybe ten people at most?

Re:Jeff Geerling – They stole my voice with AI by godelski on Sunday September 22, 2024 @02:22PM (#41618877)

  > I wonder if someone studied how unique writing style is.
Since I teach I can tell you that I can usually tell who wrote something by their language and it even works with code. There's also the Enron dataset, which is a common dataset for first time ML students where you do exactly this task.

Your language is in fact a fingerprint. And like you suggest, topics too. Much of our freedom of anonymity comes from the fact that it is hard or not worth it to dox people.

I do agree that verification is a different issue though. I'm not sure keys will solve it because you're not going to sign anything that is scandalous, so it might even give evidence for those that want to falsely claim foul play. And how do you sign a leak?

The problem with signing is that it seems to work for the cases we don't care about and do nothing for the ones we do. That is unless we sign literally everything, including our voice, but then you kill anonymity (why I connected it) and you could then probably clone that too.

Re:Jeff Geerling – They stole my voice with AI by vnorilo on Sunday September 22, 2024 @04:59AM (#41615673)
Not sure the lynch mob would pause to check that the web of trust is unbroken.
Re:Jeff Geerling – They stole my voice with AI by yreg on Sunday September 22, 2024 @06:54AM (#41616192)
I treat my accounts an non-anonymous unless I use a single-use throwaway.

I suppose even a throwaway could be linked to my identity if a comment was long enough, but probably only with some limited certainty.

Re:Jeff Geerling – They stole my voice with AI by godelski on Sunday September 22, 2024 @02:31PM (#41618956)
I treat mine as semi. I mean I'm not trying to hide from the government or anything, but yeah I'll say things I might not want to say under my clear name either. Don't we all? We all wear different masks in different groups. We all are different people in different groups too.

We're human, we all have stuff to "hide". I might want to vent about my boss or work and if they heard it that would certainly take it different than it was intended. We even complain about our friends, especially our close friends. Because no relationship is without conflict. But being human we often have to actualize our feelings with words, because we're social creatures. I'll at times complain about my partner, but that doesn't mean they also aren't my favorite person in the world.

To be human requires some anonymity. And no one should feel like they're going to be scrutinized for every little thing they say.

Re:Jeff Geerling – They stole my voice with AI by aktenlage on Sunday September 22, 2024 @05:11AM (#41615735)
Another solution would be to use an LLM to rephrase your posts, wouldn't it?

Not a great outlook though, if everybody does this...

Re:Jeff Geerling – They stole my voice with AI by rustcleaner on Sunday September 22, 2024 @01:41PM (#41618590)
I already do this in all my IRCs, Matrices, and Discords
Re:Jeff Geerling – They stole my voice with AI by godelski on Sunday September 22, 2024 @02:26PM (#41618910)
Not really. Though that will average language and poison LLMs. I'm not sure I want either of those to happen.

Besides, topics and shared details are also pii. It's more that they aren't useful in that way until you have scale.

Plus, on HN you can't edit posts after some time.

Re:Jeff Geerling – They stole my voice with AI by johnnyanmac on Monday September 23, 2024 @02:34PM (#41629149)
>I'm not sure I want either of those to happen.

at this stage and current behavior, I wouldn't mind some poison. That's the only way companies will learn at this point.

Re:Jeff Geerling – They stole my voice with AI by shevekofurras on Sunday September 22, 2024 @06:54AM (#41616193)
You can't nuke your account. You can close it but your comments remain on the site. They'll delete your account and assign your comments and posts to a random username.

Yes this violates any EU citizen's right to be forgotten under GDPR. Welcome to silicon valley.

Re:Jeff Geerling – They stole my voice with AI by whamlastxmas on Sunday September 22, 2024 @12:32PM (#41618138)
Does every comment previously attached to your account have a unique username or do they still all share the same?

Sort of moot point considering the multiple HN archives that would still have the original username attached

Re:Jeff Geerling – They stole my voice with AI by shevekofurras on Sunday September 22, 2024 @05:38PM (#41620308)
The username is changed but the comments remain attached to a single different name.
Re:Jeff Geerling – They stole my voice with AI by Der_Einzige on Sunday September 22, 2024 @01:34PM (#41618539)
How would one report this to the EU and force HN to follow the GDPR?
Re:Jeff Geerling – They stole my voice with AI by kyboren on Sunday September 22, 2024 @02:01PM (#41618720)
AFAIK Y Combinator does no business in the EU and HN is hosted in a US data center.

Under which legal theory does EU law apply here?

Re:Jeff Geerling – They stole my voice with AI by johnnyanmac on Monday September 23, 2024 @02:36PM (#41629171)
No business? It's purely an American company?

I could be hallucinating, but I swore job boards also had peopel hire from the EU.

Re:Jeff Geerling – They stole my voice with AI by godelski on Sunday September 22, 2024 @02:36PM (#41618991)

  > You can't nuke your account. You can close it but your comments remain on the site. 
Which is why I'd write an open letter and not do the thing. If I could nuke my account I wouldn't need to ask Dang, and I would have already done it.
Re:Jeff Geerling – They stole my voice with AI by kossTKR on Sunday September 22, 2024 @05:01AM (#41615684)
Yep, im sure lots of people have written a lot of random stuff on a lot of forums that should absolutely stay anonymous from gossip to family secrets to honesty about career/workplace and what not.

If stylometric analysis runs on all comments on the internet then yeah.

Bad things will happen, very very bad.

I honestly think it should be at least illegal to do this kind of analysis because it'll be a treasure trove for the commercial sector to mine this data correlated to real people not to think of the destruction in millions of people with personal anonymous blogs etc.

Actually thinking about it further you could also easily group people political affiliations, and all kinds of other thoughts, dark, dark stuff!

Re:Jeff Geerling – They stole my voice with AI by photonthug on Sunday September 22, 2024 @06:00AM (#41615983)
Idk the state of the art for forensic stylometrics or the precedents but I would be surprised if this hasn’t already been presented as evidence in many (most?) notable jurisdictions. Not definitive evidence but supporting evidence. So far from being declared illegal, it’s probably mostly established as a tool for law enforcement.

Meanwhile gangsters or nation states are probably already working on automated deanonymization and blackmail at scale, but will target the dark web before the public one. Not sure any of this stuff is that changed by the advent of deep fakes and llms tho, probably just boring classical statistics does ok?

Re:Jeff Geerling – They stole my voice with AI by greenchair on Sunday September 22, 2024 @06:45AM (#41616158)
I wonder if in this timeline it would cause people to clean up their behavior online and only post things they are comfortable having being linked back to their real identity. Would have a chilling effect.
Re:Jeff Geerling – They stole my voice with AI by godelski on Sunday September 22, 2024 @02:34PM (#41618984)
There's quite a few sci-fi books, movies, and TV shows that explore this concept. Probably the most well known is 1984

But I'm pretty sure this would make life very bland. Slow down innovation as we become less creative, less willing to deviate from the norm, and the Overton window of what's socially acceptable narrows. After all, we do love having enemies in some way or another, even if it means making them up.

Re:Jeff Geerling – They stole my voice with AI by Der_Einzige on Sunday September 22, 2024 @01:33PM (#41618527)
Stuff like this is why I tell everyone who does good work in the AI space to remember that they are watched
Re:Jeff Geerling – They stole my voice with AI by godelski on Sunday September 22, 2024 @06:04PM (#41620471)
Working in the ML space myself I try to tell my peers to think deep about their work and the consequences. Not just for ethics, but because it helps you do better.

So far I get a lot of pushback and statements like "it's just linear algebra" and "quality doesn't matter, it's quantity. Just scale. That's the bitter lesson". The former tells me they know very little linear algebra and the latter tells me they know very little about probability distribution and the concept of coverage... I routinely see papers get awards and tens of thousands of citations when they're making errors I would expect undergrads in my classes to do better at... I think a big reason I'm frustrated is that supposedly we share the same goals -- building AGI -- but I'm told I'm stopping them from doing so while they reject my papers and theirs gets accepted... (Fwiw I'm happy to accept scale papers. There should be a wide range of ideas explored. But I'm upset that we're on a railroad and any deviation from that is considered heresy)

Re:Jeff Geerling – They stole my voice with AI by rustcleaner on Sunday September 22, 2024 @01:43PM (#41618602)
>Actually thinking about it further you could also easily group people political affiliations, and all kinds of other thoughts, dark, dark stuff!

Already happening at Google and Meta for a decade or more.