Meanwhile gangsters or nation states are probably already working on automated deanonymization and blackmail at scale, but will target the dark web before the public one. Not sure any of this stuff is that changed by the advent of deep fakes and llms tho, probably just boring classical statistics does ok?
But I'm pretty sure this would make life very bland. Slow down innovation as we become less creative, less willing to deviate from the norm, and the Overton window of what's socially acceptable narrows. After all, we do love having enemies in some way or another, even if it means making them up.
So far I get a lot of pushback and statements like "it's just linear algebra" and "quality doesn't matter, it's quantity. Just scale. That's the bitter lesson". The former tells me they know very little linear algebra and the latter tells me they know very little about probability distribution and the concept of coverage... I routinely see papers get awards and tens of thousands of citations when they're making errors I would expect undergrads in my classes to do better at... I think a big reason I'm frustrated is that supposedly we share the same goals -- building AGI -- but I'm told I'm stopping them from doing so while they reject my papers and theirs gets accepted... (Fwiw I'm happy to accept scale papers. There should be a wide range of ideas explored. But I'm upset that we're on a railroad and any deviation from that is considered heresy)
Already happening at Google and Meta for a decade or more.
If stylometric analysis runs on all comments on the internet then yeah.
Bad things will happen, very very bad.
I honestly think it should be at least illegal to do this kind of analysis because it'll be a treasure trove for the commercial sector to mine this data correlated to real people not to think of the destruction in millions of people with personal anonymous blogs etc.
Actually thinking about it further you could also easily group people political affiliations, and all kinds of other thoughts, dark, dark stuff!