“Well, bad crap would be an unformatted document consisting of random letters. Good crap would be a beautifully typeset, well-written document that contained a hundred correct, verifiable sentences and one that was subtly false. It’s a lot harder to generate good crap. At first they had to hire humans to churn it out. They mostly did it by taking legitimate documents and inserting errors-swapping one name for another, say. But it didn’t really take off until the military got interested.”
To paraphrase a famous quip, what if this is proof that 99% of everything humans have created is bullshit? The problem perhaps has never been machine learning but identifying the process that enables we humans to somehow go through life and create objective societal and scientific progress out of a massive pile of nonsense.
I might be completely wrong here, but the next ChatGPT will assume this comment of mine to be absolute truth and happily ingest it wholesale.
"Politicians lie."
"Cast iron sinks."
Combine them syntactically and you get "Politicians lie in cast iron sinks", perhaps not completely false but certainly less useful than the component parts.
I realize this is a philosophical take. It also suggests a technical solution: model longer range correlations. GPTs seem so capable because previous models were not as good at this, but the context lengths modeled with transformers are still extremely short relative to those that we work with.
In result, GPTs are very good at translation and reformatting over short ranges, but struggle to deal with long form texts---a typical technique makes summaries of summaries to build out a model of a longer text. That's hardly ideal, and it's not how our minds work.
I might be completely off the mark, my knowledge of AI is very limited. But earlier today, while reading on Lisp's history and it's demise during the AI Winter, I very much enjoyed a Wikipedia article describing how symbolic AI, the main paradigm of AI research from the 60s to the 2000s, solves a completely different problem than the currently hyped neural networks—which are very good at learning, but not much more. We might need a complementary, unifying model of both approaches to reach AGI.
https://en.m.wikipedia.org/wiki/Symbolic_artificial_intellig...
The bot is incapable of intention so the entity here is still the human who prompted it. What the bullshit it produces is used for depends on their intentions.
Ironically, the only reason people get LLMs to say they hate humans and that AI is going to rise up and kill us all, is because their training data has so much text from us speculating about a robot revolution. If we'd just stayed silent about those concerns instead of always talking about robots taking over in most conversations and movies about AI, then LLMs at least would be basically incapable of even conceiving of that. So it's a bit of a self-fulfilling prophecy
On par with best available AI summary! ;) [1]
[1] https://labs.kagi.com/ai/sum?url=https://proofinprogress.com...
Maybe that’s all we as humans want, output to communicate with other humans?
This applies to my own comment/response, it is the same bullshit, may be right time will tell.
I guess my point is, what if this is exactly how new human ideas are created?
No, it hasn't. It has made great strides in certain directions, but is woefully deficient in others.
For one thing, for as impressive as it looks, it's really quite difficult to use it. The "knowledge" it has about text isn't an obviously usable understanding of the grammar and some other symbolic representation of the information that can be used by other technologies, it's more a self-referential knowledge embedded in opaque neural net values that can almost only be used to extend the text. That is not 100% true, but using it for anything else is really hard. I would expect a technology that has "solved" the language processing problem to be useful for a much wider variety of topics, to be, for instance, something I could hook up to the Unix shell to provide a safe and reliable human language interface to it.
(Note that prompting this model for some shell commands is light years from what I mean; along with the dangerous unreliability of the resulting commands, it also isn't actually hooked up to your shell and operating in a space where it knows your directories, files, and their contents. I want "give me a list of files relating to my business deals with Comcast" to literally work on the shell, to the point the result could then be piped to some other plain-text request, not what ChatGPT can do.)
"Its responses are fluent and rapidly becoming indistinguishable from human output."
Again, in some directions.
"It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality."
While I acknowledge the second clause makes your sentence accurate on your terms, I still would say it's a better way to think of LLMs that it is always bullshitting, always confabulating. It's just built in a way that the maximum-probability confabulation of a factual topic may happen to be the fact. I mean, if you think about it, that shouldn't seem that surprising a statement. I've done it myself in real life, just accurately guessed a fact I didn't really know. Less often than I've been wrong when making such guesses, of course, but it's still something not outside our experience range.
But it's not like the LLM in any sense "knows" when it is telling the truth and "knows" when it is bullshitting. You can get spun into a real philosophical tizzy about what that "knows" means, but it doesn't matter, because for any sensible definition this particular model doesn't "know". It isn't built to "know". It just spits out the maximum probability continuation. This is not a claim about all AI architectures, it's just about LLMs. They don't know. Even if you convince one to output that it doesn't know, it doesn't know that either, it just thinks that's the most likely logical continuation. While this may be surprising to anyone who has been on the Internet for a while, there are in fact samples of people saying "I don't really know" on the Internet that would have worked into the training data.
I will say this; I think GPT does serve as the final refutation of the Turing Test as a measure of AI. It turns out to be entirely possible to build an "AI" that is basically optimized to pass it, yet, strangely incapable of almost anything else we would consider an "AI" to be.
I'm actually much more impressed by GPT than I sound on HN. It is a legitimately interesting technology and very impressive. The problem is, the Gartner hype cycle has massively overshot the degree to which it is interesting and impressive, so I sound very down on it as I try to bring people back to Earth. I keep going back to the video game metaphor... video games look far more interesting and complex than they often actually are, because the graphics can look so amazing and awesome and we've made such exponential advances in that field over the past few years that we can forget that the state of the art in, say, NPC dialog, remains the "choose your own adventure" dialog tree that was a familiar staple in games for over 30 years now, with the only elaboration since then being full speech voice acting (by humans). ChatGPT is the video game graphics of the AI world. Still impressive on its own terms! It's amazing what we pump to our screens at 120Hz/4K with the right hardware... yet, what is behind those graphics is nowhere near as impressive. Similarly for ChatGPT. Very impressive! But it's a munchkin that dumped all its stat points into "sounding good to humans" but doesn't have hardly any stat points anywhere else.
Indeed, the scariest thing about ChatGPT as a technology is that the thing it is most useful for, the most obvious use case, is exactly what you are using it for: Increasing the amount of bullshit text in the world without even the gate of a human having to write it, which was already not much of a gate at all.
The problem is, that's by far the thing it is best at, and by far the most useful use for it. But that's a net loss for humanity overall. Hooray. Unfortunately, it is not something that be stopped; there's so many people rushing to take a dump in that Commons that it is unstoppable now. If indeed it hasn't already been happening for a while now (Dead Internet theory; if it isn't already true, it certainly will be).
Explain to me how this is ‘bullshit text’. It sounds to me like you are just repeating other criticisms you’ve heard because you don’t like the tool. To me, it would be rude to respond to certain emails in a way that I can tell chatGPT. I can give chat GPT a very basic framework and it will respond in a way that is not overly laconic.
Writing certain copywrite towards a requirement or proposal is also not “bullshit”. Chat got is useful for sketching out a rough outline and it’s easier to go and edit it to your liking then writing it from scratch.
I reject your definition of emails, copywrite and proposals as bullshit. They are necessary, and if I can use chat gpt to help me save time by proofreading, generating outlines, managing tone and generally making my communication more streamlined and shorter, how could you call all of that “bullshit text”.
- Summarize text: If you paste in a large amount of text, it can almost perfectly deliver a good summary. Imperfections are virtually never hallucinations; they're grammatically oddities. - Create content: As long as you're supplying the underlying facts, LLMs are great at writing blog posts/emails/etc. Intercom just released a feature that allows you to type a response and use AI to change the tone, expand it out, shorten it up, etc. LLMs are fantastic at this. - Search/information retrieval in some categories: Look, we get it, LLMs hallucinate. And of course, as many people point out, you can easily get them to hallucinate by asking specifically odd questions. But the reality is they're still really good at finding a lot of things - recipes are a great example. I've yet to see ChatGPT throw in a bizarre ingredient or omit something critical.
I think too much criticism of LLMs inaccurately assume a single use case and throw out the whole concept because they're imperfect for that use case.
I have trouble agreeing.
I've experimented with this and found that ChatGPT can produce very pedestrian prose that might pass muster on a content farm and possibly other sites with fairly low quality standards but I certainly wouldn't call it "great."
That said, it could serve as an assistant for a human writing certain types of posts.
But for other tasks the key isn't just producing believable consistent text, but something else. Perhaps the text must be factually correct or creative or traceable or have some other trait like that. And that's the majority of applications by far. And I'm sure it's the majority of monetizable applications.
I think at this point people, especially non-technical people, are lured by the glitter of ChatGPT. When it spits out an amazing limerick I immediately hear managers in my company raving about how it could revolutionize our product help documentation. They ask it a question, which it generates a correct answer to "How do you make text bold in [ourproduct]" to which it replied "You select the text and hit Ctrl+B". Manager now immediately convinced ChatGPT is intelligent enough to basically run the company. I'm trying to downplay the answer, explaining that it's more a statistical sentence-completing thing than world knowledge, and that the Ctrl+B might just have been a generic answer...
With that said, I also have an opposite argument here: that current LLMs while not intelligent, are very close to a tipping point that is going to be as important as AGI will:
The big AI revolution will be when we have AI's that people think are intelligent, not when they actually are.
There was much said about the google engineer who thought his chatbot was intelligent. And while it might not have been, we clearly saw the societal impact of AI right there. It's not what the AI can do for us, it's what ut will do TO us. Imagine if your pocket AI tamagotchi was the only being you had opened up to about something, for the last 10 years, and you felt as though it really listened. Now how would you feel if someone broke it? What happens to our communications when we can't tell artificial from real? This "societal singularity" we'd have to deal with might be a few years away, even if actual AGI is centuries away.
See also, Her (2013).
And also, Ex Machina (2014).
Yes; one of the problems seems to be that ChatGPT is playing right into some of our biggest cognitive loopholes: confirmation bias, selection bias, narrative bias. Like talking to fortune tellers, people only remember the right answers, and will silently contort the wrong ones to fit what they wanted to hear.
I do believe that AI will continue to progress, but the current form is nothing but two mirrors face to face, with us standing in the middle.
LLMs have solved the language processing problem. Its responses are fluent and rapidly becoming indistinguishable from human output. Or if anything, its responses are too good to be mistaken for an average human.
However, that’s different from producing accurate knowledge and insights. It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality.
The reference problem seems to be one example of this.