News for hackers. Stuff that matters.
Re:Are we racing toward AI catastrophe? by captainmuon on Monday February 13, 2023 @06:29AM (#34772311)
We certainly are, but it is not going to look like Terminator. We risk wrecking our society, culture, and economy.

Imagine even more power concentration in the hands of those with the required hardware to use AI. Imagine you can no longer trust any information, because the internet is being swamped by generated half-truths. An AI without conciousness, without goals, just a mastery of the language game and a poor world model. Really useful to make knowledge workers superfluous, and such that we will only see the damage long-term. And imagine all the polarisation and hysteria of social media, but x10 because the AI can close the feedback loop much faster.

If this happens we can't blame the technology, it will be entirely self-inflicted.

Re:Are we racing toward AI catastrophe? by unxdfa on Monday February 13, 2023 @06:35AM (#34772344)
This is the future I predicted years ago. I’m betting on selling myself as AI-clean because I know there is a market of people who understand the commercial, legal and societal risks associated with it. This is Pandora’s box and I don’t want to be near it.

Verified, carefully reviewed and well sourced information and data still has a place and will have a much higher value than it does now when the signal to noise ratio decreases.

Re:Are we racing toward AI catastrophe? by ianai on Monday February 13, 2023 @06:54AM (#34772462)
Might be quite the wait for the Butlerian Jihad.
Re:Are we racing toward AI catastrophe? by qbrass on Monday February 13, 2023 @07:38AM (#34772735)
11,000 years or so, canonically.
Re:Are we racing toward AI catastrophe? by unxdfa on Monday February 13, 2023 @08:53AM (#34773356)
That did come to mind :)
Re:Are we racing toward AI catastrophe? by valine on Monday February 13, 2023 @07:14AM (#34772601)
The thing is LLMs can be tested and inaccuracies corrected. In a year when the ChatGPTs of the world have API access to augment their already impressive capabilities, I doubt there will be many complaints about accuracy.

You can’t run benchmarks on an AI-clean meat sack.

Re:Are we racing toward AI catastrophe? by unxdfa on Monday February 13, 2023 @08:56AM (#34773383)
Wait until people start training AI on AI output because (a) that's the only source of information left with a large enough corpus size and (b) all other sources are polluted.

There's no way of asserting accuracy when we reach that point because the very nature of information and scientific thinking will be corrupted.

Re:Are we racing toward AI catastrophe? by valine on Monday February 13, 2023 @09:52AM (#34774091)
There is evidence to suggest that LLMs have recently passed a threshold where their performance can be improved by training on their own output.

https://arxiv.org/abs/2210.11610

I expect your concerns are likely unwarranted.

Re:Are we racing toward AI catastrophe? by danaris on Monday February 13, 2023 @03:32PM (#34779965)
How does this do anything but validate their concerns?

They're not saying "if we train LLMs on their own output, they'll get slower or worse", they're saying "if we train LLMs on their own output, it gets even harder to actually derive sourcing for what they produce."

Re:Are we racing toward AI catastrophe? by pegasus on Monday February 13, 2023 @01:35PM (#34777996)
That's what proponents of self-driving cars keep promising also.
Re:Are we racing toward AI catastrophe? by unxdfa on Tuesday February 14, 2023 @03:12PM (#34795299)
Self crashing cars please.
Re:Are we racing toward AI catastrophe? by ajsnigrutin on Monday February 13, 2023 @06:33AM (#34772329)
> Imagine even more power concentration in the hands of those with the required hardware to use AI. Imagine you can no longer trust any information, because the internet is being swamped by generated half-truths.

You already cannot trust anything, because it's being generated with humans with political agendas.

> An AI without conciousness, without goals

Better without goals than with goals that threaten the people.

The only thing that will rise is the level of noise compared to usable data... and with seo-optimized articles, there has been a lot of noise already.

Re:Are we racing toward AI catastrophe? by kevviiinn on Monday February 13, 2023 @06:34AM (#34772336)
We haven't been able to trust things on the internet for a long time

"Who would go on the internet and tell lies??"

Re:Are we racing toward AI catastrophe? by pegasus on Monday February 13, 2023 @01:33PM (#34777975)
And yet... you ain't seen nothing yet.
Re:Are we racing toward AI catastrophe? by guestbest on Monday February 13, 2023 @02:16PM (#34778704)
Except this a lot broader than telling lies. AI can mimic faces and voices in a way that actors are currently incapable. Our lie making capabilities have just been surpassed
Re:Are we racing toward AI catastrophe? by jcutrell on Monday February 13, 2023 @06:35AM (#34772351)
A better picture is an AI without guardrails that would be generated by ethical goals.

An AI without goals is functionally equal to an AI with nefarious or unethical goals - boundless action trained on human language / behavior will likely skew bad.

Re:Are we racing toward AI catastrophe? by GauntletWizard on Monday February 13, 2023 @06:35AM (#34772346)
* * *
Re:Are we racing toward AI catastrophe? by peoplefromibiza on Monday February 13, 2023 @06:40AM (#34772387)
so, basically Ted Kaczynski was right?
Re:Are we racing toward AI catastrophe? by MrLeap on Monday February 13, 2023 @06:53AM (#34772459)
The internet is already a swamp of half truths. They're just predominately, for now, generated by meat. I tend to be on the side of meat though so I'm still with you.

"we will only see the damage long-term" -- I'm not confident this is true or false. I can see a few things ahead of us though!

Here's my predictions:

1. The most profitable AI will be a webapp version of a hostess club that aggrandizes, validates and flatters for cash. At some point it'll get too good at this and humans will have to step in to moderate it so it bankrupts fewer people. It'll get warning labels.

Within this model will be really sinister truths about the human condition that will be hard to prize out.

2. In the short term, there will be a few really wonderful works of art created that will benefit from the force multiplication AI tools provide.

Re:Are we racing toward AI catastrophe? by Rzor on Monday February 13, 2023 @07:01AM (#34772507)
If it gets bad enough where info on the web is completely unreliable we might see people reverting back to TV and media empires in search of "trust" and "vetted info". The internet would be treated as poisoned. Still an utterly grim picture. But this betrayal would have to be mighty.

This is not going to happen, isn't it? Even if riddled with chatbots is still persuasive enough to generate engagement. Some could choose dark corners, but for the majority would be business as usual.

Speaking of business as usual, how the fuck the ad business could survive this? The only way out that I can see is it if IDs are forced on us somehow, but that's a shitshow in itself.

Re:Are we racing toward AI catastrophe? by MrLeap on Monday February 13, 2023 @07:24AM (#34772655)
The era of the "human info curator" is probably a natural evolution of the influencer era. It would be interesting if snapshots of Wikipedia from now-ish end up being seen as the ultimate corpus of unadulterated truth, before the AI's corrupted it. That'll probably happen at the fringes. Everything that can happen does happen at the fringes. Further ingress is the question.
Re:Are we racing toward AI catastrophe? by kneebonian on Monday February 13, 2023 @07:02AM (#34772520)
> Imagine you can no longer trust any information, because the internet is being swamped by generated half-truths.

So like the world was about 20 years ago?

Re:Are we racing toward AI catastrophe? by postexitus on Monday February 13, 2023 @07:03AM (#34772527)
AI is not any different than a bunch of people working together towards something. AI is the people. People are the AI. Just imagine a corporate entity - with little nodes running around making neurologic synapses and making decisions all based on their statistical understanding of the past etc.
Re:Are we racing toward AI catastrophe? by arpa on Monday February 13, 2023 @07:05AM (#34772535)
hey chatgpt we need a spin on how ai is actually human
Re:Are we racing toward AI catastrophe? by mro_name on Monday February 13, 2023 @07:07AM (#34772551)
> not going to look like Terminator

we're approaching Brazil.

Re:Are we racing toward AI catastrophe? by squishy47 on Monday February 13, 2023 @07:11AM (#34772579)
I've been thinking for a few years that books are going to (re-?)become a more viable way of obtaining infomation as the internet becomes the "dark forest" (https://maggieappleton.com/ai-dark-forest) and the effort of filtering the trash outweighs the cost and time of books. Maybe we might engineer and innovate our way back to a time before instentanious knowledge, who knows.
Re:Are we racing toward AI catastrophe? by marban on Monday February 13, 2023 @07:16AM (#34772609)
My only hope: People's ability to judge over synthetic content will increase and therefore the free market will correct and eventually wipe out BS-generating factories while at the same time, the value of verified human creation and trusted/old brands is going to increase. It's going to be a qualitative dip until we get there.
Re:Are we racing toward AI catastrophe? by 2OEH8eoCRo0 on Monday February 13, 2023 @08:11AM (#34772967)
Hardware is the easy part. The tricky part is acquiring the immense troves of information that's required. This is the moment that the surveillance capitalists have been waiting for.

I see some hope though. In-person communication becomes more valuable again. Established media with editorial review and reporters on the scene becomes valuable again. Software freedom gets a boost with more intelligent reverse engineering/translation.

Sites like YouTube also won't sit idly by while untold millions of AI generated garbage videos are uploaded and not viewed.