News for hackers. Stuff that matters.
Re:The AI bullshit singularity by scottbez1 on Sunday February 18, 2024 @03:19PM (#39422725)
Succinctly stated and something that resonates strongly with me.

In the last internet revolution (web search), results started high quality because the inputs were high quality - bloggers and others just wanted to document and share knowledge. But over time, many interests (largely commercial) figured out how to game the system with SEO, and quality of search results has decreased as search's incentive structure led to lower quality data being indexed.

We're at the start of the LLM revolution now - models are trained on high quality inputs (which may be as rare as "low-background steel" in the future). But the models allow the mass production of lower quality outputs with errors and hallucinations; once those get fed back into new models, are we doomed to decreasing effectiveness of LLMs, just as we've seen with search? Will there be LLMO (LLM optimization) to try to get your commercial interests reflected in the next generation models?

I think we've got a few golden years of high quality LLMs before that negative feedback loop really starts to hurt like it did in search.

Re:The AI bullshit singularity by daxvena on Sunday February 18, 2024 @04:13PM (#39423398)
I doubt it's going to go the same way as search. You can't run Google on consumer hardware, but you can run LLMs locally.

At worse, newer models will get worse and you can just stick to older models.

You could also argue that proprietary models gated by an API are better than anything you can run locally, and yeah maybe those will get worse with time.

They're not going to get any worse than what you can run locally though. If they do, open models will overtake them, and then we'd be in a better position overall.

Re:The AI bullshit singularity by Jensson on Monday February 19, 2024 @05:54AM (#39428332)
> You can't run Google on consumer hardware, but you can run LLMs locally.

You can't run an up to date model locally. When I ask Googles models they have knowledge from stuff just a week ago, without using search. You wont get that from a giant local model.

Re:The AI bullshit singularity by rocqua on Sunday February 18, 2024 @03:40PM (#39423026)
Unlike the low background radiation steel, high quality content will still continue to be created. Arguably even at a faster rate with these powerful tools. The proportion will drop ofcourse but that only means curation will become king. With search this curation was difficult, because the value was so low you could only do it profitably if it was completely automatic, and that was difficult.

An optimistic take would be that LLMs will make curation so much more valuable, that it will be done much better. If the wider world gets to use this curation to limit spam, and highlight good work, it would be amazing for the world.

Re:The AI bullshit singularity by ofrzeta on Sunday February 18, 2024 @04:19PM (#39423476)
Let the crawler AI discard a sample if it detects it as AI generated? That's at least quite expensive I think.
Re:The AI bullshit singularity by hermitcrab on Sunday February 18, 2024 @04:17PM (#39423457)
>LLMs will make curation so much more valuable, that it will be done much better.

Won't that also mean a return to a world of gatekeepers?

Re:The AI bullshit singularity by slowmovintarget on Sunday February 18, 2024 @04:32PM (#39423595)
That gatekeepers will be the ones deploying the ML systems.
Re:The AI bullshit singularity by rocqua on Monday February 19, 2024 @03:31PM (#39434454)
Yes, hopefully they will be good.
Re:The AI bullshit singularity by slowmovintarget on Sunday February 18, 2024 @04:31PM (#39423580)
You're forgetting that we have mechanisms in place to curate curation and farm attention toward the crap, not toward quality. It will be harder and harder to break the surface tension of this gooey gelatin wrapper we've placed over creative activity.
Re:The AI bullshit singularity by MichaelZuo on Sunday February 18, 2024 @08:20PM (#39425256)
People will pay money for curation towards quality, probably quite a bit if the rest of the landscape is 99.99% noise. They won't pay much money for curation "toward the crap".

So I'm not nearly as pessimistic.

Re:The AI bullshit singularity by slowmovintarget on Sunday February 18, 2024 @11:37PM (#39426264)
But that will be reputation-based, and names can be sold. Look at the brand-holding conglomerates we have today that don't make any of the original product, they just license the name out to anyone, regardless of the quality of the finished good. From mattresses to magazines, we keep seeing this. Why wouldn't we see this in curation sites?
Re:The AI bullshit singularity by MichaelZuo on Sunday February 18, 2024 @11:44PM (#39426303)
Sure, there will always be some fraction of dupes and double games being played, but how does that change the main point?

It's very unlikely for 100% of all intelligent curators to become dishonest either so it doesn't seem like a serious roadblock.

Re:The AI bullshit singularity by Nuzzerino on Sunday February 18, 2024 @03:34PM (#39422935)
The comparison is only valid if AI ends up being monopolized. A continuously evolving ecosystem on the other hand has a better chance of adapting to those pressures. I am sure there are search engines that don’t index the SEO crap, but I don’t remember the names, and they have other flaws.

Open source AI needs to get a lot of investment for this to be mitigated. Relying on market incentives to drive development without the possibility of forking is too dangerous given the high stakes.

Re:The AI bullshit singularity by piva00 on Sunday February 18, 2024 @04:00PM (#39423282)
Given enough time, everything seems to go the way of consolidation. I don't have much hope that AI will be different...
Re:The AI bullshit singularity by TaylorAlexander on Sunday February 18, 2024 @03:37PM (#39422974)
I think it’s clear that LLMs cannot be the end state of this technology, and we will need systems that can reason and develop hypotheses and test them internally. These systems may benefit from more curated datasets (such as those collected before the bullshit wave began) along with real world interaction data from YouTube and robotics. Such systems could eventually be used to rank web pages for their bullshit level, which of course would present a risk of censorship but when used properly could lead to insightful data handling.

It’s just really clear that a giant text averaging machine can only go so far, and while we do see some higher level emergent properties, we’re going to have to move beyond the current state of the art in the next decade, and such future systems may be much less affected by the internet’s bullshit. Even without the bullshit singularity I think such measures would be a necessity and we would see them developed soon.

Re:The AI bullshit singularity by pton_xd on Sunday February 18, 2024 @03:43PM (#39423068)
> It’s just really clear that a giant text averaging machine can only go so far

It's not really a text averaging machine, it's a pattern matching machine.

Right now the "depth" of the patterns it can match can only go so far, but in a few years with more advances in chips and memory the depth is going to increase and the patterns it can match will fan out accordingly.

Re:The AI bullshit singularity by vrighter on Monday February 19, 2024 @01:47AM (#39426833)
it is a statistical model. If everyone is saying X and is wrong, and one guy says Y and is right, the llm will bit out X. Because that is the most probable thing in the dataset. It literally is a text averaging machine
Re:The AI bullshit singularity by visarga on Sunday February 18, 2024 @03:47PM (#39423123)
Not just real world data from videos. AI models need feedback from many sources: humans, code execution, web search, simulations, games, robotics, math verification, or from actual experiments in the real world. All of these are environments that can take the output of a model and do some processing and return feedback. The model can learn and search for solutions, creating its own training data as a RL agent.

Since all deployed models produce some kind of effect and feedback from the world there is an opportunity there to collect data targeted on the current level of the model, the most useful kind of data. That's why I think AI will be ok even with the proliferation of bots online. It's not 100% pure synthetic data in a loop, it is a agent-environment loop.

tl;dr Models learn better from their own experiences, not ours.

Re:The AI bullshit singularity by noduerme on Monday February 19, 2024 @04:53AM (#39427850)
But bullshit hallucinogenic output and fakery at colossal scales doesn't just pollute the pool of static information available on the web. It also warps the minds of the humans you're relying on to verify reality on the next training loop. Not only that, but we can also see the emergent breed of humans who believe that writing is similar to arithmetic - an unnecessary skill that can be handed off to a calculator. Or that making a movie shouldn't require knowing anything besides asking for what you want to see. How is someone like that - someone who wants to rely on bots - going to tell a bot what is or isn't true? How can they even have a pre-pollution baseline understanding of reality?

I just finished rereading Do Androids Dream for the first time in 20 years or so, and was astonished at how similar his andys really are to LLMs in the polluted / destroyed reality there. How confusing and corrupting they are to organic life. PKD describes the androif brains as neural networks with thousands of layered pathways and trillions of weighted parameters, and it's as if he was able to accurately conceive of what linguistic and "emotional" strengths and weaknesses those constructs would actually have, decades before LLMs existed. And there's this one amazing line where Deckard calls them "Life thieves". What else should we call what Sam Altman and others are building - but theft of human ingenuity, creativity, and basic reason for living, and the utter annihilation and suppression of people like this 16 year old kid who dare to hope they can contribute something more original in life than being a servant of a tech company building this shit, or a tiktok influencer who writes prompts?

What should that kid hope: That their work becomes noticeable enough to be immediately stolen and their name turned into a prompt?

Life thieves.

Just as a further aside, I had dinner tonight with a friend who's a fairly famous animator in the commercial realm, and I brought up this post. He's just sure the kid's screwed and the genie is out and creative is basically over. He's turning to building wooden clocks.

But his reaction, and the reactions I see here every time this comes up, remind me of something else. They remind me of how people react when someone is robbed. Everyone has some reason why it's bad but not that bad, it was inevitable, it'll be okay, etc. Or they go around wondering what they're going to do now. Or they paper it over with optimism. Surprisingly few people get robbed and are willing to realize they were robbed, and become wildly pissed off about it. Most people have some sort of flight reaction, as evidenced by e.g. the promotion of "prompt writing" or learning to use a paid API to do what was formerly your own creative job that now spits your own work back at you.

I say sue the shit out of all these content thieves.

Re:The AI bullshit singularity by __loam on Sunday February 18, 2024 @03:38PM (#39423000)
We may enter into a highly ironic regime where ai companies subsidize writers and artists at scale to produce better data.
Re:The AI bullshit singularity by aj7 on Sunday February 18, 2024 @03:47PM (#39423133)
I like this idea very much. Is this similar to human moderation of training data?
Re:The AI bullshit singularity by __loam on Sunday February 18, 2024 @03:52PM (#39423185)
Sure. I suspect it might be hard to actually recruit people like this considering how the people who got sampled are hugely pissed off at AI firms rn, but money will go a long way here.
Re:The AI bullshit singularity by spaceguillotine on Sunday February 18, 2024 @04:31PM (#39423584)
Royalty agreement could make this work, every time their data gets used, they get a cut, it would be the only way i could see people agreeing to this.
Re:The AI bullshit singularity by Turing_Machine on Sunday February 18, 2024 @03:40PM (#39423021)
I guess some of it might depend on how good the AI-generated content gets, and also how good the AI gets at detecting AI-generated content.

If the AI was good at detecting it, it wouldn't matter if the AI-generated content sucked, yes?

Even a low probability of detection would help. Let's say our algorithm is 50% likely to detect AI junk. That means that half the junk data won't make it in to model. Even 20% would probably be worthwhile, especially if it also threw away human-generated junk (and let's be realistic here: there is, and always has been, no shortage of terrible and/or wrong human-generated content).

Let's say those crappy filler paragraphs that get stuck between pictures on meme clickbait pages... I suspect most of those people have already been replaced, but that prose was horrible long before the current LLM boom.

I suspect there is a lot of effort being expended right now on ways to ensure the training data (whether human or AI generated) isn't shite.

Re:The AI bullshit singularity by vrighter on Monday February 19, 2024 @01:49AM (#39426841)
ai detectors can be used to train ais to avoid triggering them. It's a race that cannot be won. ai detection can't, and already doesn't, work.
Re:The AI bullshit singularity by Turing_Machine on Monday February 19, 2024 @10:37AM (#39430912)
> ai detection can't, and already doesn't, work.

Then how can humans often tell that something is generated by AI?

Besides, I wasn't suggesting AI detection as such. I was suggesting crap detection, regardless of how the crap was produced.

You probably don't want crappy human-generated content going into your training set, either.

Re:The AI bullshit singularity by asadotzler on Monday February 19, 2024 @07:19PM (#39436510)
Humans are smarter than machines. We "sense" bullshit in patterns that machines cannot yet intuit. I can tell when something's AI generated. It has a smell or a flavor that's different from human work and it's a variable pattern because, like a southern accent, I can tell if it's Georgia or Alabama, Gemini or Claud, even if very capable machines cannot.
Re:The AI bullshit singularity by Solvency on Sunday February 18, 2024 @03:36PM (#39422961)
This comment/sentiment/idea has already been thoroughly discussed, shared, retweeted, posted, and echoed a couple thousand times online already. Humans like you have been redundantly saying the same things in varying degrees of novelty, like this, for centuries.
Re:The AI bullshit singularity by alkonaut on Sunday February 18, 2024 @03:52PM (#39423182)
That's what the topic of the post we are discussing is. He is basically reiterating the point. Which is a good point. And there have been zero good arguments made (as far as I'm aware) to why this fear would be unfounded.
Re:The AI bullshit singularity by Solvency on Sunday February 18, 2024 @04:07PM (#39423355)
I'm pointing out the irony. The Internet is already saturated with us continually metaphorically JPG compressing original ideas into textual redundancy, with many people having only been exposed to those copies of copies.
Re:The AI bullshit singularity by hermitcrab on Sunday February 18, 2024 @04:22PM (#39423508)
I'm not surprised this isn't an original idea. But I haven't seen it discussed.
Re:The AI bullshit singularity by lm28469 on Sunday February 18, 2024 @04:22PM (#39423506)
And "humans like you" have been saying what you just said for the same amount of time, what's your point?

If we can only dicuss ideas which have never bee shared we can close this website right now

Re:The AI bullshit singularity by hackerlight on Sunday February 18, 2024 @07:33PM (#39425019)
I don't think it impacts AGI timelines much. Worst case scenario, we just cut off training at 2024 data. But it's not like another ~5 years of internet data is the magic that will get us over the finish line to AGI. We should have enough data already. We will get to AGI with synthetic data (e.g. AlphaGeometry for code, or simulators e.g. RL inside unreal engine), world model from video, algorithm improvements, and 100x more compute.

I'd add that majority of LLM-generated stuff put onto the internet isn't pure junk. It's endowed with human-created context and curation. If someone submits some LLM-created code to Github, it's because it works. A small fraction is pure noise (e.g. state-actors spamming Twitter) but that should remain a minority.