Which kinds of organizations don't care about correctness or reputation? Scammers, spammers, and certain types of propaganda-spewing organization, especially those that puppeteer "grassroots" campaigns. We may give a single-digit multiplier in productivity to some roles in legitimate business, while giving a two or three digit multiplier to similar roles in harmful, parasitic organizations.
Some countries have already successfully rewritten the history of less than 30 years ago with surveillance and media control.
What will happen when they can also control all digital reality? They can invent an uprising of millions of people that never actually happened, and there will be photo, audio, and video "evidence".
AGI is scary, but it's not immediate. I think companies like OpenAI are wrecklessly pushing us toward immediate societal collapse.
Nuclear technology is tricky to get working even if you have decent scientists and all the published research about basic principles. For current ML tech, a big point is that the core techniques are really simple, the simple textbook methods work at large scale, you don't necessarily need any special sauce which could be restricted. Yes, there is a lot of engineering work that goes into a full system to scale it efficiently, but all of that can be replicated by any decent bunch of engineers, unlike nuclear weapons.
The cat is out of the bag; you can't put the genie back in the bottle, all the "sensitive knowledge" you'd need to restrict AI is already known by at least a few CS undergrads in every single university in the world.
A given AI, once created, is "just" data. It can be sent around the world and run on commodity hardware. Nothing special needed beyond the expertise that created it in the first place.
Think of how relatively easy it is to clamp down on bioweapons, and compare that with our ability to stop malware.
Certainly the models and weights that have been released publicly are out in the wild and near impossible to retract. That damage is done.
Soviet union was doing it back in the early 1900s
Even the ancient world did it https://en.wikipedia.org/wiki/Damnatio_memoriae
Yes we can fake more, but there are also more ways to get information so its harder to suppress true info. I think the two cancel each other out.
"More" is dramatically understating it. The new platforms (GPT4, Midjourney, etc.) are able to automate the propaganda output of literally millions of people.
If you wanted someone to Photoshop the pope into multiple new outfits and fool a lot of people, you used to need at least a few hours of an expert's time. Now you can do it almost instantly.
It's many orders of magnitude faster to create highly realistic lies than it used to be.
> but there are also more ways to get information so its harder to suppress true info.
There are the same ways to get information, and they're going to be: A) unverifiable, and B) drowned out by propaganda.
Let's say someone wants to take down Joe Biden, and they generate text, audio, video, and photos of him molesting a child. Millions of people would want to believe it and then believe it. How would we prove it's not real? How could we, other than finding the person who did it or exposing a digital trail?
They don't, unfortunately. The continued influence effect means that information received first will generally take priority.
I would characterize this as just adding more and more landmines to an already crowded space of potential collapse-inducing events through which have to try to navigate society. Oh, and adding glitzy blinkenlights to attract people towards those landmines. "Let the marketplace decide if landmines are fun to step on."
Very succinct way to explain end stage capitalism.
looks at history of organized religion
Um, I'm not so sure about that...
It doesn't matter if they're cautious or not. Even for real media, how can you tell it's real?
The result will be:
- there will be no way for anyone to verify any digital media, including phone calls from loved ones
- people will believe what they want to believe
We've seen the second thing happening, and it leads to fascism.
You've unfortunately been doing these things repeatedly, and they're not what the site is for and destroy what it is for.
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
What is the public gonna do?
Oh, I see; goto school, goto work, passively consume todo lists society foists on them to distract from exploring their individual dynamism in deference to empowering billionaires to build AI they hate.
Maybe if the masses logged off rather than blithely contribute, but you all got fiat wealth to hallucinate the conjuring of in deference to elders who don’t know you; they just been told it’s your job to prop up their past investments. Pitter patter.
Otherwise enjoy living in a figurative ouroborus pushing Altman and the like towards the mouth, yourself towards the other end, then die some day. Enjoy that strange loop!
None of them have divine authority to the resources the species relies on. They’re surrounded by meat based VHS tapes repeating the truisms the elder’s foisted upon us decades ago.
Indeed. And, as every other AI project has shown us, the rough areas are easy, the last few percent are incredibly difficult.
Look at self driving cars. It will easily be 2050 before we can even hope to have good, driving AI. Current versions are a joke, and, are being trialed in places like California.
Not on snow, dirt roads, while snowing, on rural roads, on unmapped roads, of which there are plenty.
2050 at least.
Chatgpt will be vastly improved in maybe 30 years. The last little bit will take decades.
At what point will the skeptics just give in? This shit is already working. It’s a lawyer’s and a businessman’s problem not a technical.
You truly think we need 27 years to support “rural roads” because of technical issue?
A self-driving car that can safely drive in all the same places and conditions that humans can is not yet possible.
This describes any "creative" job, where the hard part of the job is not rejecting the bad ideas but coming up with the good ones. Visual art is one famous example. Programming is another: if your code doesn't pass the test suite, you can tell, and then you can fix it. Some software needs higher assurance than test suites can provide, and then you need to get your code to pass Rust's borrow checker, or a more rigorous proof checker like ACL2. Telling a valid proof from an invalid one is a solved problem; constructing a program you can write a valid proof for is very much not.
Programmers and visual artists are also famously cavalier about correctness and reputation, being willing to try anything, including notorious bohemian lifestyles.
As far back as the discovery of fire, new technology has enabled more positive outcomes than negative.
Some tech more so than others, I guess, but what makes you lean so negatively regarding AI? It’s already improved my life considerably. Just the basic ChatGPT web app has extended my capacity in multiple respects.
For example, the Internet is nothing like TV, so arguments like "we grew up on TV and ended up ok" don't hold.
Same holds in my opinion to AI-scams.
> we grew up on TV and ended up ok
Not sure what you are comparing here, but we (people in early 30s) also grew up on the internet and mostly ended up ok. Some people who grew up on tv did not end up ok.
If I catch a fish in a lake that's not problem.
If I catch all the fish in the lake we need to stock the lake again with fish.
If I catch 10^30 fish on the planet, fish are extinct.
The entire planet is extinct
https://www.ucsusa.org/resources/how-nuclear-weapons-work
> Modern nuclear weapons work by combining chemical explosives, nuclear fission, and nuclear fusion. The explosives compress nuclear material, causing fission; the fission releases massive amounts of energy in the form of X-rays, which create the high temperature and pressure needed to ignite fusion.
Gen III nuclear reactors like those being installed at Vogtle and other sites area 21st century design.
Is fusion weapon research informative of fusion reactor research, and vice versa?
The nearest nuclear weapon is around 800km from my location, so you could say they're pretty effective at this.
That being said I don't believe it's mainly the nuclear weapons that let me live in peace, but I like to believe they're a component of that.
Since their inception they have revolutionized modern biology, and by extension medicine [1]. Also they gave us the seismic monitoring network we currently use to understand the earth.
Of course it wasn't the weapons that did this, but rather the offshoot technologies. But like every technology the first use was blowing something up, the later uses were arguably more peaceful.
I would agree that these days there's a bit more divergence between weapons research and useful nuclear research. But there are still some things that might belong in both camps (e.g. inertial confinement fusion).
But yea, not nukes themselves, just the technology and innovations that they made along the way... were the real treasure?
It may be in your nature to question everything you see, hear or read. I'm pretty sure that's not the case for the vast majority of population. And as magical ChatGPT seems to all the tech and non-tech people, it is really difficult to predict what it will look like in 30 years from now. It may now take 100 trolls to spread fake news about Ukraine on Twitter.
In 5 years from now it might take one prompt and you will be able to cover every single social media on the planet in convicing information from many spectrums of the opinions. What you might see could be 10 people violently arguing on the internet, whilst its all bots whose only purpose is incite emotions and polarize people.
We know that this already happens, but now its in the grasp of many people and thus much harder to pin down where its coming from and what that party's motive is. How can you 'follow the money' when - it requires almost no money to spam every social media channel - then get picked up by news bc they turn to social media for fast/new/relevant information
But we have been here before. When radio first came out, it had an almost magical sway on people. People attribute the newness of radio as a significant factor in hitler's rise to power. Eventually we adapted to it though, and it lost its magic sway.
I interpret your opening statement here positively. I tend to think most people work towards the good of mankind, so a boost in productivity for everyone is a net positive.
Systems like Chat GPT and other large models do not exist in nature that we know of.
AI can be copied and can integrate with a lot of existing infrastructure right out of the box. Combine this with the magnitude of the capabilities and our societal "immune system" doesn't stand a chance of being ready for whatever is about to emerge. That lack of time to learn and adapt is a massive difference.
Yes we are. We are approaching a world where people will not want to invest their time, energy, and emotion engaging with other supposed humans remotely unless they have verified their personhood / identity.
I'm actually surprised we have not yet seen an entire industry built around human authentication. Seems like Apple is the only company taking this seriously.
The standard of human interaction will either be meeting IRL or signing communications biometrically.
Completely agree - but to that - with how quickly AI voice cloning is improving, and how quickly image creation tools are improving, how long will it be until someone can sample my voice well enough to command Siri to do something and Siri won't be able to tell the difference? How long will it be until an AI generated photo of me is so realistic that it can unlock my phone via FaceID?
I have a feeling that's going to happen in a shorter timeframe than we all think it will. I mean, my voice is somewhat similar to my father's, and if I'm around his phone, I can say "Hey Siri" and his phone will think it's his voice, not mine.
In the future, we might be only able to unlock our phone or command Siri when hooked up to a continuous biometrics sensor.
Perhaps that's Apple's angle with the AR/VR headset it supposedly has in the works - if you've got your headset on, you can unlock your devices, and if you don't, you have to manually unlock devices via a password or PIN or whatever.
Most young people these days aren't even using phone numbers/SMS these days except when forced to, so the threat will remain with the older generation and either die off with them or until the phone companies solve this problem... Or some digital voice system replaces it.
Otherwise it's just email phishing. Anything involving money transfers is already locked down and mostly a matter of fooling people... Which again is mostly older people with a poor grasp of technology and common threats (modern street smarts).
If you want a technical solution, you have to demand that a person always contact you from their own device as "2FA", or at least that they some kind of 2FA device on them... except there are a billion ways that somebody might lose access to such a device when they were in genuine trouble, and scammers are totally capable of making it look like one of those scenarios has happened.
You're really down to the point where people have to do "mind to mind" authentication with shared secret knowledge... while under extreme stress... in a case that's uncommon enough that most people will not have practiced it.
This is not so great.
Not sure enough to feel really comfortable betting her life on it, anyway. Not if we hadn't drilled it on a daily basis for weeks and a weekly basis for months.
It's easy to blank on things when you're adrenalized, say if you've been kidnapped. And it's also easy to blank on things when you're adrenalized because you are hearing the person's voice saying they've been kidnapped.
... and if I asked a scammer pretending to have kidnapped her to let me call her on her phone, I would expect to get the obvious reply: "I threw her phone away. I'm not dumb enough to let you track me/her/us through it". Which is totally credible because that's what a kidnapper should do.
When you get the call, the strong prior probability is that the whole thing is a scam, but that's not so easy to hold onto in a situation like that. And even if you do hold onto it, you will be scared.
Oh, and on edit: Yes, I expect I would keep it together enough to call her on her phone to check, since if she hasn't been kidnapped there's nothing stopping her from answering it. I don't know if I'd expect that of others. But it's also true that if I call her, and nothing is actually wrong, I still expect about a 50-50 response rate because she doesn't hear the thing, has it on mute, or is in school and forced to keep it in her locker, or has let the battery run down, or whatever.
I don't think it will be long before video and audio is no more convincing than text is now. We will stop falling for the AI scams, just like we (most of us) stopped falling for the Nigerian princes, scam ads/virus popups on the web, and those fake emails from family members claiming they need us to wire money so they can pay for a flight home or whatever. Basically, my thesis is that people will start to get wise to any scam that is sufficiently common and harmful.
Real kidnappers might have to learn to work to convince the families that the kidnapping is real and not just another scam.
Every piece of communication will have to be signed to verify authenticity.
Camera makers are already looking into embedding authentication technology at the hardware level.
It's an inevitability.
If it's someone you know, with a reputation that makes them believable, then that's basically a criminal act and depends heavily on that person's reputation. There should be enough deference even without that, beyond simple harassment (which is what you're saying, where you lose your job or relationships).
Otherwise I'm not sure why your spouse or boss would rather believe some video coming from [email protected] is more believable than the person themselves.
Anyway, I was talking about overt scams not the wider cultural/social implications. This is a large new burden that will be imposed on society. We will learn to adapt and society will manage.
It's impractical to have authenticity enforced by millions of consumer devices. If any random manufacturer in a third-world country can generate millions of valid signing keys to be embedded in millions of cheap phones or cameras, then there are many employees there that can also leak a bunch of valid keys if the price is right, and these keys can sign anything in a way that's indistinguishable from all these cheap cameras. And if you revoke the signatures of any "untrustworthy" manufacturers (really, any manufacturer can be and will be compromised, especially if some governments want to manufacture propaganda), then people won't stop using the phones/cameras just because their signatures aren't considered kosher, you'll still have millions of people uploading genuine, valid images from these cameras, so either people will have to trust invalid signatures or refuse lots of benign content, and I'll bet most people will choose the former one.
And of course a camera doesn't know if it's taking a picture of reality or another picture - it would take a relatively simple optical setup to allow any camera to record (and sign) an image off of another screen, so if some authenticity-verification system was actually working and popular, any reasonable criminals would do it and have signed recordings of their deepfakes from a real major brand camera.
It feels horrible, invasive and like the state has outsourced it to a third party you dont know and have no control/direct engagement with.
Every single time historically that we've had this level of authentication, there is a 'strong man' type that uses this information against the population. Whether its simple strong societal shame in smaller communities or its a massive program to track and/or persecute a group of people.
We were past it years ago in some domains.
I used to do a lot of language exchanges, and there is/was a need to screen for people who are solely using machine translation. It's pointless correcting someone using machine-generated output.
From a convenience standpoint, it seems like we've gotten so technologically advanced that we are starting to move backwards. Is there a name for this type of phenomenon?
The physical world is what we are fully integrated into as embodied beings along dimensions we do not yet and may never understand. It is the only space that has an inherently scarce quality, which seems to be something we need as humans. IMO technology should enable us to deepen our connection in the "real world," not isolate us in piss poor representations built on the belief that we are merely mind-bearing mechanical vessels to be manipulated.
AI is going to drive us back to fully-embodied living, and that's good.
Um, you do see the problems with using biometrics, right?
Isn't Sam Altman involved in some Crypto company trying to collect biometrical data, using some orb thing, in exchange for worthless crypto? No idea if he still is, but I am more and more convinced that some folks took every single cyber punk story not as a satire or warning, but as a playbook and something to strife for.
This is fairly low-tech and likely susceptible to various kinds of social engineering, but I'm curious what a more robust approach might look like that doesn't involve us all regurgitating 6-digit codes like robots all the time.
I tell you I'll email you immediately to verify. I email you to confirm you want N $X gift cards. You do so.
Or if the inbound contact is by email, you verify by text or phone call. Or Signal/Discord/FB chat apps, etc. Heck even LinkedIn Inmail could be your verification channel.
You could still be compromised, but that'd require attackers to have significantly more access and readiness. And if you reach the other person and they're unaware, now you both have some support as you figure it out.
Software is often aimed at solving problems and making processes more cost-effective. Founders are often most interested in solving their own problems, problems of their friends, or the most valuable problems of their customers. For B2B companies to customers are other businesses. The greatest cost for businesses are employee costs. Solving the employee cost, but eliminating employees, is a great target for business owners.
It's not like fake news was a non-issue before ChatGPT existed. Breitbart, and other fake news sites existed for years before this was even imaginable. Fake graphics were around for ages too, with image manipulation, even before computers. Take for example the Surgeon's Photo of the Loch Ness Monster.
The arena of the continuous, which encompasses most of the natural world, is far more difficult. It doesn’t allow for the sort of arguments where one can assert things with ego-boosting absolute confidence.
Matters of scale naturally fall into the continuous sort, but scale can also introduce new discrete possibilities. Perhaps we need to present the new specific, discrete effects and outcomes that AI-at-scale will introduce. Maybe, just maybe, that’ll change minds of people who are truly open to being changed.
My intuition though is that this argument is repeated here (on HN in particular) not for lack of knowledge or thought, but rather because they WANT to see the chaos that’ll result. They want all the positives and negatives, no matter the balance, simply because it’s exciting and adds to their “mundane” lives. Where the mundanity is of course subjective, simply the result of their default worldview, rather than anything nearing an objective description of the world as it is.
It's certainly exciting but I wouldn't bet on it adding to our lives just yet. Maybe. But many avenues leading to net negative outcomes are still part of the tree.
A normal person couldn’t use photoshop to detect the image was a fake. But a normal person will be able to use AI to detect it.
I see all negative nancies about it. I don’t see anyone realizing that as good as the tools are at faking, the tools to detect will be exactly as good.
[citation needed]
We might as well hit the problem Google Translate hit, where the training set started to contain more and more data created from GT itself. Similar thing may happen with your NN. At some point there may be so much AI-generated content (by different AIs) that it may be difficult to compose a trustworthy training set.
I'm worried we may reach a point AI will get so good faking people that real peoples' output will be treated as fake due to only as many combinations possible that originates from humans. You will start depending on AI telling you what is true and what is fake for every single piece of information. This leads to a question how to tell which AI is right - you can't really verify an opinion, only facts.
And being able to manipulate opinions is a very strong perk.
I'm skeptical content generation was the thing holding back this wave of fake news and scams from ruining the Internet. These doomer posts are always handwavy on the specifics.
And we've seen little evidence of it operating at scale in the past 2yrs since these tools have been unleashed.
Maybe automated niche targeting but that still depends on networks with reputation systems + spam detection... So mostly depends on ostensibly solvable tech problems like caller ID spoofing, or someone sending mass email spoofed campaigns.
That you have seen little evidence of it doesn't mean that it isn't happening, it could also mean that (1) you're looking in the wrong places, (2) that it is so good that you don't detect it and probably other explanations besides those.
Otherwise it's mostly just predictions of widescale disruption which beyond hyper targetted attacks I remain skeptical.
Search "gmail spam" on hn.algolia.com with a "past year" time filter, here's a bunch of the most popular titles:
- Bypassing Gmail's spam filters with ChatGPT
- Ask HN: What's happening with Gmail spam filtering?
- Tell HN: Gmail's spam filters have gone bonkers
- Gmail spam is out of control
- Ask HN: Gmail having spam problems for anybody else?
- Ask HN: Have you noticed an increase of spam evading Gmail's filters recently?
- Ask HN: Has Gmail's spam filter been defeated by this type of spam lately?
Maybe it's not relentlessly posted every week but a lot of people are certainly noticing it on HN.
I disagree...
I create 'fake news' message among a series of bots and have a strong network there of social media accounts and sites. Ok, that's step one.
Now, I hack your account and steal your reputation. It appears you approve this message and some subset of your followers follow the bots and go to the bot sites.
You can try to pull out and say your hacked, and some percentage of people will unfollow said bots, but at the same time, a subset of those followers are now following those bots and giving them thumbs up thereby getting access to your friends friends.
User accounts are easily hackable as we saw with Linus Tech Tips recently. Reputation is just the newest currency worth stealing.
Maybe even the opposite, perhaps the deluge of AI-generated content will make the average person trust less what they find at random sources... which is healthy.
Note that in general I'm not too optimistic about AI risks (the very news that motivated this thread is scary) but I don't see the worry about mass misinformation in particular to be such a big deal.
https://astralcodexten.substack.com/p/sorry-i-still-think-i-...
Was atleast trying to look legit. I think OP means like blogspam news scam shit.
The bad-faith commenters, YouTube, journalists and other types who have an axe to grind already happily cite garbage sources without verifying (or perhaps caring) when making their motivated arguments and there's more than enough out there to back up whatever BS they're trying to spew any minute. I don't see how quantity available changes that. And of course AI can be deployed in the counter direction. I think (hope) you need a qualitative change in order to tip the balance of power.
Tech evolved any idiot can get a copy of photoshop and some video software, we need tp solve the problem, otherwise is like preventing creation of email because we are too incompetent to address the spam problem.
Managed properly, it could lead to what is essentially a utopia.
YES
In fact, not approaching, but already passed the threshold.
If you have any assets, it'd time to be sure you have a set of actually obscure un-guessable prompt & responses to verify identity in your family. And it needs to be better than "what's our first dog's name?" or "where do we vacation?" type stuff found on FB; if they're going to the trouble of getting images & voice samples to clone, they'll find that stuff.
Now, it is not that everyone is famous for 15 minutes, it is that everyone needs to be up-to-date on security to avoid being randomly shot or scammed. Nice society we've built.
As it stands, people believe whatever dumb thing they read online. Thanks to AI, they will have to learn that if something is out of the ordinary, they should confirm with multiple sources. I call this a net win.
And for those literally medically incapable of this level of reasoning, we will soon have "AI firewalls" (Gibson ICE) that can tell people "Hey, this looks like a scam!" and also help them reason about complex topics.
This is fantastic, what a time to be alive!
I once shared this approach, but learned that the majority of people don't really want to leave their echo chambers and gladly accept anything that amplifies it. And the bad part about it is that the western world is based on democracy, and democracy is controlled by the majority. You may not care about people that are easy to manipulate, but in reality you are not the target as your voice won't matter at scale.
We are already there. In the kidnapping context, we have been there for a great many years. If someone say they have kidnapped my child, I will text/call that child immediately to verify. The 'second factor' is the realworld daughter. A kidnapper must both create a facsimile of the daughter and then also render the real daughter incommunicado. We don't need new 2FA because we already have it.
Almost as quickly, AI may unwind all of that. It lay become that there's more noise than signal across all of these technical media. The only thing you can trust, and the only thing that matters, is the people in front of you.
Just like the old days.
I'm hopeful that if AI understands language as well as a super-intelligent human, it will also be able to detect "new human thoughts". This would be the perfect newsfeed.
And if it can't detect "new human thoughts", then we've reached AGI and I'm so proud to be alive for it.
However, I'm not sure if total loss of pseudonymity is less of a horror scenario.
Smaller, more trusted rooms, will combat the threat of misinformation.
The universe is built on waves/cycles. I think we are going to see pressures to make smaller rooms because of AI.
I'm actually excited for smaller, more intimate spaces, with trusted people who now have 10x ability from the powers of AI. That seems like a LOT of fun.
I don't think we truly know just yet how negative an impact AI will have on society. Every time there has been a technological leap, people have panicked over what the gizmo of the now will do to society. Again, not an invalid concern, but society has yet to have been blown apart by anything.
Also, everything potentially being just AI may inadvertently get the public to do the right thing. People should never have been as trusting of authority figures or institutions in the first place. If everyone assumes that everything is likely to be complete hogwash, which was already true in many cases, then they may not just swallow everything as fact. Maybe fewer people will blindly consume news, which is a good thing; 99% of news is not actionable or good for an average person's well-being. And if enough ransom demands turn out to be AI-generated scams such that the real ones are overall far less successful, it's possible that fewer people are kidnapped for ransom in the first place.
I'm not saying I know any of this, but rather the opposite.
I thought I was immune to certain scams where the scammer accent was a dead giveaway that they were scammers. Now with capabilities of AI I feel somewhat vulnerable again though Im far from being elderly.
You'd hear people from out of town talking about what they saw in a neighboring city. You'd need to judge how trustworthy the person was. People would expect a chain of narration, to understand how _that_ person came to learn a bit of information (or if they claimed to witness it first hand).
As AI generated content becomes more popular, I predict we as a society are going to go back to relying more and more on the reputation of the speakers in question.
Who might you consider reputable? People you've met personally, people who your community respects, and of course, influencers who you follow who's persuasive words match your pre-existing world views
We can have parties again and have friends instead of depression.
1. A not exactly rocket scientist POTUS takes office in 2024. Let's suppose they were a populist, triumphalist, religious, agro, law-and-order DINO to make it interesting and fictional.
2. Rogue AI launches a social engineering communication offensive against CENTCOM impersonating generals, cols, and ltcs against enlisted and lower ranks to carry out a first strike against China with a digital hallucination that Taiwan is being "attacked". It uses details gleaned by wiretapping the upper echelon to circumvent normal N-person keying rules and authentication protocols.
WarGames 2: Joshua Wins
Now, the likes of DALL-E and DeepFake can generate convincing fake graphics. Chatgpt and the likes generate convincing fake news. Voice AI can generate convincing voice from small samples.
If you were afraid of your elderly relatives being scammed by people pretending to be policemen or grandchildren, now more tech-concious people will get scammed by the voice and look of their relatives. Are we really approaching the reality where we need 2FA to trust the other person is really who they are?