News for hackers. Stuff that matters.
Re:The AI Crowd Is Mad by jimbokun on Wednesday February 01, 2023 @09:08AM (#34610578)
Is this a good summary?

LLMs have solved the language processing problem. Its responses are fluent and rapidly becoming indistinguishable from human output. Or if anything, its responses are too good to be mistaken for an average human.

However, that’s different from producing accurate knowledge and insights. It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality.

The reference problem seems to be one example of this.

Re:The AI Crowd Is Mad by soneca on Wednesday February 01, 2023 @09:16AM (#34610691)
People were worried that if we created an AI on our image, it would be a belligerent entity. But we did create an AI on our image, and it’s a bullshitter.
Re:The AI Crowd Is Mad by arethuza on Wednesday February 01, 2023 @09:36AM (#34611001)
"Artificial Inanity" as Neal Stephenson called it in Anathem:

“Well, bad crap would be an unformatted document consisting of random letters. Good crap would be a beautifully typeset, well-written document that contained a hundred correct, verifiable sentences and one that was subtly false. It’s a lot harder to generate good crap. At first they had to hire humans to churn it out. They mostly did it by taking legitimate documents and inserting errors-swapping one name for another, say. But it didn’t really take off until the military got interested.”

Re:The AI Crowd Is Mad by sph on Wednesday February 01, 2023 @09:28AM (#34610901)
On a philosophical level, I wonder if ChatGPT gives us mostly nonsense because most human output is bullshit.

To paraphrase a famous quip, what if this is proof that 99% of everything humans have created is bullshit? The problem perhaps has never been machine learning but identifying the process that enables we humans to somehow go through life and create objective societal and scientific progress out of a massive pile of nonsense.

I might be completely wrong here, but the next ChatGPT will assume this comment of mine to be absolute truth and happily ingest it wholesale.

Re:The AI Crowd Is Mad by Jensson on Wednesday February 01, 2023 @09:36AM (#34611003)
The average of two true statements can easily become false, so models like ChatGPT will spout nonsense even if it was only trained on 100% truthful statements.
Re:The AI Crowd Is Mad by james-revisoai on Wednesday February 01, 2023 @12:32PM (#34614161)
Can you give an example as to what you mean please?
Re:The AI Crowd Is Mad by b3morales on Wednesday February 01, 2023 @01:27PM (#34615116)
One classic example from linguistics is the two (true) sentences

"Politicians lie."

"Cast iron sinks."

Combine them syntactically and you get "Politicians lie in cast iron sinks", perhaps not completely false but certainly less useful than the component parts.

Re:The AI Crowd Is Mad by thedorkknight on Wednesday February 01, 2023 @10:12AM (#34611539)
It gives us nonsense because it's goal is to generate something that looks like it fits in with training data. You can have training data that is all completely factual and still have a generator spit out nonsense that looks completely correct without fact-checking.
Re:The AI Crowd Is Mad by naasking on Wednesday February 01, 2023 @10:27AM (#34611796)
So, exactly like people then?
Re:The AI Crowd Is Mad by inciampati on Wednesday February 01, 2023 @10:30AM (#34611839)
A key problem is that the transformer operates over very short fragments of text. Long range correlations need to be maintained to ensure 100% accuracy in reproduction/inference of relationships.

I realize this is a philosophical take. It also suggests a technical solution: model longer range correlations. GPTs seem so capable because previous models were not as good at this, but the context lengths modeled with transformers are still extremely short relative to those that we work with.

In result, GPTs are very good at translation and reformatting over short ranges, but struggle to deal with long form texts---a typical technique makes summaries of summaries to build out a model of a longer text. That's hardly ideal, and it's not how our minds work.

Re:The AI Crowd Is Mad by sph on Wednesday February 01, 2023 @12:08PM (#34613652)
That's a good observation, though I'm not sure it's just a matter of longer range correlations, i.e. fixable by merely adding more computational power. The truth problem is fundamentally different than what those neural networks are trying to solve. We got the learning part almost right, we have made no progress towards understanding.

I might be completely off the mark, my knowledge of AI is very limited. But earlier today, while reading on Lisp's history and it's demise during the AI Winter, I very much enjoyed a Wikipedia article describing how symbolic AI, the main paradigm of AI research from the 60s to the 2000s, solves a completely different problem than the currently hyped neural networks—which are very good at learning, but not much more. We might need a complementary, unifying model of both approaches to reach AGI.

https://en.m.wikipedia.org/wiki/Symbolic_artificial_intellig...

Re:The AI Crowd Is Mad by orbital-decay on Thursday February 02, 2023 @02:13AM (#34623231)
Sure, a lot of human output is incoherent, but current models give nonsense for a different reason - they lack many mechanisms typical for human intelligence.
Re:The AI Crowd Is Mad by recuter on Wednesday February 01, 2023 @09:20AM (#34610781)
Seems to me it is more of a babbler. You know, like a young toddler just learning to talk.
Re:The AI Crowd Is Mad by dandellion on Wednesday February 01, 2023 @10:58AM (#34612299)
Or a million monkeys with typewriters, only these monkeys are not entirely random, they have been conditioned to write things similar to what people actually write. Most of the time the output looks very natural, but others you get nonsense because the noise is too loud.
Re:The AI Crowd Is Mad by api on Wednesday February 01, 2023 @09:17AM (#34610716)
Hook a gun up to it and you have a belligerent bullshitter. Seems fairly accurate for a good bit of humanity.
Re:The AI Crowd Is Mad by JohnFen on Wednesday February 01, 2023 @10:17AM (#34611624)
Isn't a bullshitter a belligerent entity?
Re:The AI Crowd Is Mad by alpos on Wednesday February 01, 2023 @10:33AM (#34611891)
Belligerence is an intention. Bullshitting is a technique.

The bot is incapable of intention so the entity here is still the human who prompted it. What the bullshit it produces is used for depends on their intentions.

Re:The AI Crowd Is Mad by JohnFen on Wednesday February 01, 2023 @04:24PM (#34618004)
I think belligerence can also be a technique and bullshitting can also be an intention.
Re:The AI Crowd Is Mad by thedorkknight on Wednesday February 01, 2023 @10:18AM (#34611632)
ChatGPT is only non-beligerent because it's biased with prompt preface that tells it it's not belligerent. Raw LLMs will absolutely spit out racist, violent garbage and threaten you.

Ironically, the only reason people get LLMs to say they hate humans and that AI is going to rise up and kill us all, is because their training data has so much text from us speculating about a robot revolution. If we'd just stayed silent about those concerns instead of always talking about robots taking over in most conversations and movies about AI, then LLMs at least would be basically incapable of even conceiving of that. So it's a bit of a self-fulfilling prophecy

Re:The AI Crowd Is Mad by freediver on Wednesday February 01, 2023 @09:37AM (#34611012)
> Is this a good summary?

On par with best available AI summary! ;) [1]

[1] https://labs.kagi.com/ai/sum?url=https://proofinprogress.com...

Re:The AI Crowd Is Mad by duxup on Wednesday February 01, 2023 @09:30AM (#34610928)
> It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality.

Maybe that’s all we as humans want, output to communicate with other humans?

Re:The AI Crowd Is Mad by rileymat2 on Wednesday February 01, 2023 @09:21AM (#34610785)
I kind of feel like that is exactly what your comment is, you synthesized a lot of other material and are throwing it out there for analysis. We go through a lot of bad ideas that humans believe that are basically bullshit.

This applies to my own comment/response, it is the same bullshit, may be right time will tell.

I guess my point is, what if this is exactly how new human ideas are created?

Re:The AI Crowd Is Mad by jerf on Wednesday February 01, 2023 @09:51AM (#34611208)
"LLMs have solved the language processing problem."

No, it hasn't. It has made great strides in certain directions, but is woefully deficient in others.

For one thing, for as impressive as it looks, it's really quite difficult to use it. The "knowledge" it has about text isn't an obviously usable understanding of the grammar and some other symbolic representation of the information that can be used by other technologies, it's more a self-referential knowledge embedded in opaque neural net values that can almost only be used to extend the text. That is not 100% true, but using it for anything else is really hard. I would expect a technology that has "solved" the language processing problem to be useful for a much wider variety of topics, to be, for instance, something I could hook up to the Unix shell to provide a safe and reliable human language interface to it.

(Note that prompting this model for some shell commands is light years from what I mean; along with the dangerous unreliability of the resulting commands, it also isn't actually hooked up to your shell and operating in a space where it knows your directories, files, and their contents. I want "give me a list of files relating to my business deals with Comcast" to literally work on the shell, to the point the result could then be piped to some other plain-text request, not what ChatGPT can do.)

"Its responses are fluent and rapidly becoming indistinguishable from human output."

Again, in some directions.

"It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality."

While I acknowledge the second clause makes your sentence accurate on your terms, I still would say it's a better way to think of LLMs that it is always bullshitting, always confabulating. It's just built in a way that the maximum-probability confabulation of a factual topic may happen to be the fact. I mean, if you think about it, that shouldn't seem that surprising a statement. I've done it myself in real life, just accurately guessed a fact I didn't really know. Less often than I've been wrong when making such guesses, of course, but it's still something not outside our experience range.

But it's not like the LLM in any sense "knows" when it is telling the truth and "knows" when it is bullshitting. You can get spun into a real philosophical tizzy about what that "knows" means, but it doesn't matter, because for any sensible definition this particular model doesn't "know". It isn't built to "know". It just spits out the maximum probability continuation. This is not a claim about all AI architectures, it's just about LLMs. They don't know. Even if you convince one to output that it doesn't know, it doesn't know that either, it just thinks that's the most likely logical continuation. While this may be surprising to anyone who has been on the Internet for a while, there are in fact samples of people saying "I don't really know" on the Internet that would have worked into the training data.

I will say this; I think GPT does serve as the final refutation of the Turing Test as a measure of AI. It turns out to be entirely possible to build an "AI" that is basically optimized to pass it, yet, strangely incapable of almost anything else we would consider an "AI" to be.

I'm actually much more impressed by GPT than I sound on HN. It is a legitimately interesting technology and very impressive. The problem is, the Gartner hype cycle has massively overshot the degree to which it is interesting and impressive, so I sound very down on it as I try to bring people back to Earth. I keep going back to the video game metaphor... video games look far more interesting and complex than they often actually are, because the graphics can look so amazing and awesome and we've made such exponential advances in that field over the past few years that we can forget that the state of the art in, say, NPC dialog, remains the "choose your own adventure" dialog tree that was a familiar staple in games for over 30 years now, with the only elaboration since then being full speech voice acting (by humans). ChatGPT is the video game graphics of the AI world. Still impressive on its own terms! It's amazing what we pump to our screens at 120Hz/4K with the right hardware... yet, what is behind those graphics is nowhere near as impressive. Similarly for ChatGPT. Very impressive! But it's a munchkin that dumped all its stat points into "sounding good to humans" but doesn't have hardly any stat points anywhere else.

Re:The AI Crowd Is Mad by rhodorhoades on Wednesday February 01, 2023 @09:57AM (#34611301)
Try some prompt engineering courses. It’ll change your perception. Some of my prompts are like 1200 words just to set context. But I use it for email, marketing outreach, blogs, you name it. It’s drastically increased my productivity as a solo consultant.
Re:The AI Crowd Is Mad by jerf on Wednesday February 01, 2023 @09:59AM (#34611342)
That wouldn't even remotely change my impression. I'm already aware, as I said, that ChatGPT is very good at continuing text.

Indeed, the scariest thing about ChatGPT as a technology is that the thing it is most useful for, the most obvious use case, is exactly what you are using it for: Increasing the amount of bullshit text in the world without even the gate of a human having to write it, which was already not much of a gate at all.

The problem is, that's by far the thing it is best at, and by far the most useful use for it. But that's a net loss for humanity overall. Hooray. Unfortunately, it is not something that be stopped; there's so many people rushing to take a dump in that Commons that it is unstoppable now. If indeed it hasn't already been happening for a while now (Dead Internet theory; if it isn't already true, it certainly will be).

Re:The AI Crowd Is Mad by eternalban on Wednesday February 01, 2023 @11:02AM (#34612381)
ChatGPT ~is Auto-tune for text. It already kinda sucked to be poor but it is going to really suck going forward.

https://en.wikipedia.org/wiki/Auto-Tune

Re:The AI Crowd Is Mad by rhodorhoades on Thursday February 09, 2023 @05:43PM (#34732770)
I write around 100-200 emails a day, coordinate across various projects and write a lot of copywrite around the solutions and systems the company develops.

Explain to me how this is ‘bullshit text’. It sounds to me like you are just repeating other criticisms you’ve heard because you don’t like the tool. To me, it would be rude to respond to certain emails in a way that I can tell chatGPT. I can give chat GPT a very basic framework and it will respond in a way that is not overly laconic.

Writing certain copywrite towards a requirement or proposal is also not “bullshit”. Chat got is useful for sketching out a rough outline and it’s easier to go and edit it to your liking then writing it from scratch.

I reject your definition of emails, copywrite and proposals as bullshit. They are necessary, and if I can use chat gpt to help me save time by proofreading, generating outlines, managing tone and generally making my communication more streamlined and shorter, how could you call all of that “bullshit text”.

Re:The AI Crowd Is Mad by dwaltrip on Wednesday February 01, 2023 @11:39AM (#34613078)
Do you have some courses you recommend?
Re:The AI Crowd Is Mad by idopmstuff on Wednesday February 01, 2023 @09:51AM (#34611215)
I think a lot of people (like OP) seize on the problems with using LLMs for search while ignoring the many things that it can already do more or less flawlessly.

- Summarize text: If you paste in a large amount of text, it can almost perfectly deliver a good summary. Imperfections are virtually never hallucinations; they're grammatically oddities. - Create content: As long as you're supplying the underlying facts, LLMs are great at writing blog posts/emails/etc. Intercom just released a feature that allows you to type a response and use AI to change the tone, expand it out, shorten it up, etc. LLMs are fantastic at this. - Search/information retrieval in some categories: Look, we get it, LLMs hallucinate. And of course, as many people point out, you can easily get them to hallucinate by asking specifically odd questions. But the reality is they're still really good at finding a lot of things - recipes are a great example. I've yet to see ChatGPT throw in a bizarre ingredient or omit something critical.

I think too much criticism of LLMs inaccurately assume a single use case and throw out the whole concept because they're imperfect for that use case.

Re:The AI Crowd Is Mad by ghaff on Wednesday February 01, 2023 @10:44AM (#34612083)
>As long as you're supplying the underlying facts, LLMs are great at writing blog posts

I have trouble agreeing.

I've experimented with this and found that ChatGPT can produce very pedestrian prose that might pass muster on a content farm and possibly other sites with fairly low quality standards but I certainly wouldn't call it "great."

That said, it could serve as an assistant for a human writing certain types of posts.

Re:The AI Crowd Is Mad by alkonaut on Wednesday February 01, 2023 @10:00AM (#34611353)
I think so yes. I think it has solved the problem of making believable human language with some sense of consistency. That's no small feat, and for some tasks (writing limericks, reformulating a letter, translation) it's even the whole point of the exercise, so we have AI making something a "Solved problem" to the point that we had it recognize digits better than humans, years back. No small feat.

But for other tasks the key isn't just producing believable consistent text, but something else. Perhaps the text must be factually correct or creative or traceable or have some other trait like that. And that's the majority of applications by far. And I'm sure it's the majority of monetizable applications.

I think at this point people, especially non-technical people, are lured by the glitter of ChatGPT. When it spits out an amazing limerick I immediately hear managers in my company raving about how it could revolutionize our product help documentation. They ask it a question, which it generates a correct answer to "How do you make text bold in [ourproduct]" to which it replied "You select the text and hit Ctrl+B". Manager now immediately convinced ChatGPT is intelligent enough to basically run the company. I'm trying to downplay the answer, explaining that it's more a statistical sentence-completing thing than world knowledge, and that the Ctrl+B might just have been a generic answer...

With that said, I also have an opposite argument here: that current LLMs while not intelligent, are very close to a tipping point that is going to be as important as AGI will:

The big AI revolution will be when we have AI's that people think are intelligent, not when they actually are.

There was much said about the google engineer who thought his chatbot was intelligent. And while it might not have been, we clearly saw the societal impact of AI right there. It's not what the AI can do for us, it's what ut will do TO us. Imagine if your pocket AI tamagotchi was the only being you had opened up to about something, for the last 10 years, and you felt as though it really listened. Now how would you feel if someone broke it? What happens to our communications when we can't tell artificial from real? This "societal singularity" we'd have to deal with might be a few years away, even if actual AGI is centuries away.

Re:The AI Crowd Is Mad by bongobingo1 on Wednesday February 01, 2023 @10:31AM (#34611859)
> Imagine if your pocket AI tamagotchi was the only being you had opened up to about something, for the last 10 years, and you felt as though it really listened. Now how would you feel if someone broke it?

See also, Her (2013).

Re:The AI Crowd Is Mad by codetrotter on Wednesday February 01, 2023 @11:48AM (#34613268)
> See also, Her (2013).

And also, Ex Machina (2014).

Re:The AI Crowd Is Mad by b3morales on Wednesday February 01, 2023 @12:04PM (#34613579)
> I'm trying to downplay the answer, explaining that it's more a statistical sentence-completing thing than world knowledge, and that the Ctrl+B might just have been a generic answer...

Yes; one of the problems seems to be that ChatGPT is playing right into some of our biggest cognitive loopholes: confirmation bias, selection bias, narrative bias. Like talking to fortune tellers, people only remember the right answers, and will silently contort the wrong ones to fit what they wanted to hear.

I do believe that AI will continue to progress, but the current form is nothing but two mirrors face to face, with us standing in the middle.