Please don't reduce LLM down to ChatGPT (or generative models more generally). People are using LLM for real-world problems every day. BERT and its descendants/variants are used all over the place for many different problems in natural language processing. I and my team have used it on dozens of different projects, mainly in classifying text documents and inputs. And it works very well. Multilingual LLMs are responsible for the huge improvements in machine translation; my team has to deal with text in multiple languages, and these models are vital there too. We have used LLM on real-world problems that are in production now and are saving hundreds of person-hours of tedious work.

ChatGPT? Yeah, it's neat. I'm sure people will find some useful niche for it. And I do think generative models will eventually have a big impact, once researchers find good ways to ground them to data and facts. This is already an active area of research -- combining generative LLMs with info retrieval methods, or targeting it to a specific context. (Meta just gave a talk last week at the NeurIPS conference about teaching a model to play Diplomacy, a game that mostly involves talking and negotiating deals with the other players. ChatGPT is too broad for that -- they just need a model that can talk about the state of the game board.) So in general, I'm optimistic about generative LLMs. But ChatGPT...is just a toy, really. It's not the solution -- it's one of the signposts along the way toward the real solution. It's a measure of progress.

I wouldn't undersell ChatGPT. It's like a repl for a particular LLM. Maybe there are others but it's the first time many people have gotten direct access to the technology. Sometimes the medium is the message.

That's fair -- perhaps we could frame it as a large-scale beta test of sorts. Researchers are building LLMs to solve problems, but new technologies can often end up solving problems they were never designed for. Once people get their hands on them, they test and tinker and find new uses for them. Sometimes it turns out not to be a good solution to the initial problem, but a great solution for something completely different.

For instance, while I'm still generally of the opinion that generative models have limited use unless they're grounded to reality...I did see a post on Reddit about someone using ChatGPT to generate story ideas for their D&D game. So yeah...don't need to be tethered to reality to make a fantasy story! That's not something I would have thought of (even though I'm a DM!), and it's still relatively niche, but it's a great story of how getting something into people's hands to play with can generate lots of new ideas.

Do you feel like for some tasks though that people might be too eager to reach for GPU and more expensive inference and training costs?

In some projects I'm working on, I'm seeing close to 95F1 score using deep learning to do token classification based NER. However, using non-deep learning approaches (still training statistical models) I can get to 91F1 on my use case but have a much faster inference and not need to use a GPU.

I'm similarly optimistic about LLM, but fear that a lot of other really useful workhouse algorithms and strategies are going to be pushed aside and people will forget about them.

Back to my use case, some strategies I'm looking at are using cheap and fast CPU powered models/smaller models for inference and then based on certain signals decide whether a particular instance should be passed to a GPU based model for better accuracy.

I don't know if I'm too concerned about that, to be honest. Yeah, there's a huge cost in terms of training the LLMs, and then there can be a cost for downstream inference, but I think it depends on the use case. In some cases, performance is the absolute top priority; in other cases, you might be willing to trade off some performance for better inference time, or model size, etc. If you need to put the model on cell phones or offline low-power devices, that's a key constraint that might make you reach for a different tool.

The nice thing about more "classical" approaches -- a simple BoW random forest or MLP, for example -- is that they're typically quick to train and experiment with, and they make for great baselines, if nothing else. So I doubt that we're in danger of people forgetting about them entirely. If people do, they're leaving quick, easy solutions on the table.

I do like your idea about triaging inference between smaller CPU vs. larger GPU models based on whatever signals. I haven't tried that before, but a project my colleagues worked on did some triaging between regex pattern-matching vs. model inference. Basically, the regex pulled some of the data out first if it matched very specific, known patterns, and then the rest was handled probabilistically. I guess the effectiveness of that sort of triaging approach depends on how strong and clear your signals are that let you choose one path over the other.

LLMs may be overhyped, but transformers in general are underhyped.

LLMs make a lot of mistakes because they don't actually know what words mean. The key thing is though - it's much harder to generate coherent text when you don't know what the words mean. In a similar vein it's completely unreasonable to expect an LLM to perform visual tasks when it literally has no sense of sight.

The fact that it can kind of sort of do these things at all is evidence of the super-human generalization potential of the transformer architecture.

This isn't very obvious for English because we have prior knowledge of what words mean, but it's a lot more obvious when applied to languages humans don't understand, like DNA and amino acid sequences.

How can these things not know what words mean? Did you not see how they created a virtual machine under chatGPT? They told it to imitate bash and they typed ls, and cat jokes.txt and it outputted things completely identical to what you'd expect. Look it up. https://www.engraved.blog/building-a-virtual-machine-inside/

I don't see how you can explain this as not knowing what words mean. It KNOWS.

The whole input of your current session is fed into the model, that's why it tricks you that it "KNOWS" like human would know, in reality this is a lot of data, computation and statistics without any reasoning, that's why there is a lot of examples showing it contradicts itself in the same paragraph, because it doesn't know the meaning of the words its using in the same way as humans do, it only knows probabilities of words in sequences.

LLMs are trained exclusively on text, which means they lack crucial context behind the meaning of sentences. The universe of information outside of pure text - vision, sound, etc is completely unknown to it.

LLMs are basically the aliens in blindsight. They have a superhuman ability to memorize the context of words it has seen and generalize to new contexts, but it can never be perfect because it's working on incomplete information.

> it can never be perfect because it's working on incomplete information.

Unlike you?

Yeah, that's the actual bit that baffles me about ChatGPT still. Producing coherent, fluent text is alright, but we could already sort of do that 20 years ago with markov models or even just grammars (see Chomsky).

Understanding text in the depth that ChatGPT (and GPT-3) appear to understand the prompts is something entirely different and has to my knowledge never been archieved before the current architectures.

There is a lot of knowledge encoded into the model, but there's a difference between knowing what a sunset is because you read about it on the internet vs having seen one.

> This will hardly seem like a controversial opinion, but LLM are overhyped.

As the [excellent] paper points out, LLMs are complex functions that can be embedded in systems to provide plausible answers to a prompt. Here's the money sentence.

  LLMs are generative mathematical models of the statistical distribution
  of tokens in the vast public corpus of humangenerated text, where the tokens 
  in question include words, parts of words, or individual characters including 
  punctuation marks. 

Rather than focus on the limitations of this approach to answer general queries, which are manifest, it seems more interesting to ask a different question. Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans? The answer would:

1. Illuminate where we can use LLMs safely.

2. Direct work to make them better.

It's already impressive that within certain scopes ChatGPT gives very good answers, indeed better than most humans.

> Under what circumstances do LLMs give answers that are reliably equivalent to or better than humans?

Which humans? Humans give a... fairly wide range of responses.

I'm a history major and love classical history. My first question to ChatGPT was:

> Why was Julius Caesar murdered?

The answer was the following, which would pass a standard exam question on the topic. It exhibits [the appearance of] multi-layer reasoning and has a nice conclusion.

> Julius Caesar was assassinated on the Ides of March in 44 BC by a group of Roman senators who believed he had become too powerful and posed a threat to the Roman Republic. The senators were concerned that Caesar had ambitions to become king, which would have ended the Republic and given him absolute power. The senators were also concerned about Caesar's growing popularity with the people of Rome. They believed that assassinating him was the only way to prevent him from becoming too powerful and destroying the Republic.

It's interesting to note that most of the evidence for this answer including 2000 years of interpretation is available in textual form on the Internet. It's easily accessible to LLMs.

Average humans? Within 1 stdev from the mean?

> Average humans? Within 1 stdev from the mean?

This implies that performance has unqiue natural, objective, ratio-level (or at least, a unique consistent interval-level) measure. Otherwise the mean is, itself, meaningless.

“How well you answer a question” doesn't seem to fit that, its maybe at best a (somewhat subjective, still) ordinal quality, so the median (or mode) is the only useful average.

But I think you'll find that without restricting things more than “humans”, both the median and mode of responses to most prompts is... quite bad.

This reminds me that coding with ChatGPT felt like pair programming with a not super smart person who could google and type really fast. Not really fun!

I disagree that LLMs are overhyped, but it's very subjective. Are current LLMs a few steps from AGI? No. Will LLMs change the computing landscape? Yes, I believe they will.

ChatGPT, without any major changes, is already the best tool out there for answering programming questions. Nothing else comes close. I can ask it to provide code for combining two APIs and it will give useful and clean output. No need to trudge through documentation, SEO-hacked articles, or 10 different Stack Overflow answers. Output quality will only improve from here. Does it sometimes make mistakes? Yes. There are also mistakes in many of the top SO answers, especially as your questions become more obscure.

Aside from programming, how many other fields are there where LLMs will become an indispensable tool? I have a PhD and ChatGPT can write a more coherent paragraph on my thesis topic than most people in my field. It does this in seconds. If you give a human enough time, they will be able to do better than ChatGPT. The problem is, we're already producing more science within niche scientific fields than most scientists could ever read. As an information summary tool, I think LLMs will be revolutionary. LLMs can help individuals leverage knowledge in a way that's impossible today and has been impossible for the last 30 years since the explosion in the number of scientific publications.

It can reproduce a statistically plausible paragraph, certainly. But there is a great deal more to research than producing statistically plausible paragraphs. It doesn't _understand_ anything!

I've actually worked on a project where there have been attempts to use GPT like models to summarize scientific results and the problem is it gets shit wrong all the time! You have to be an expert to separate the wheat from the chaff. It operates like a mendacious search engine pretending to be a person.

The problem is that we need to pair generative models with verification systems. We have the models, but no verification yet. Fortunately code and math are easier to verify. Some things require simulation. In other cases you can substitute an ensemble of solutions & picking the most frequent answer as consistency based verification. But for each domain we need to create verifiers and that will take some time.

The good thing is that we'll be able to generate training data with our models by filtering the junk with the verifiers. Then we can retrain the models. It's important because we are getting to the limit of available training data. We need to generate more data, but it's worthless unless we verify it. If we succeed we can train GPT-5. Human data will be just 1%, the race is on to generate the master dataset of the future. I read in a recent paper that such a method was used to improve text captions in the LAION dataset. https://laion.ai/blog/laion-5b/

>we need to pair generative models with verification systems >code and math are easier to verify

I would love to see a two-stage pipeline using a LLM to convert natural language specifications into formal specifications for something like Dafny, and then follow up with another model like AlphaZero that would generate code & assertions to help the verifier. This seems like something that a major group like DeepMind or OpenAI could pull off in a few years.

One concern here is that if ChatGPT replaces the need to go to websites like Stack Overflow or Wikipedia, what happens to them? Do they stick around if the only people who visit them are there to feed new stuff to chatGPT? Also, how does chatGPT get hold of papers and articles behind pay walls? How much of the scientific publications are free?

I disagree and think this is a very controversial opinion.

Playing around with it last night convinced me that LLM's are a huge, game changing technology. I was trying to decide which material to use for an upcoming project. The model doesn't use the internet without some hacking, so I had it write a program in python using the tkinter UI kit.

I asked it to create a UI with input boxes for material, weight of material, price and loss due to wastage. The program takes all of those inputs and converts the material into grams from KG, pounds, ounces. It then calculates the price per gram and takes a loss percentage (estimate given by user). It then writes a text file and saves it to a directory.

I literally pasted the code into VS code and had to change Tkinter to tkinter. Hit run and it worked flawlessly. I have NEVER used tkinter and it took about 30 minutes from start to finish.

This morning, I asked my 9th grade son what he is learning in 9th grade biology. He told me he is learning cellular endocytosis. I asked chapGPT to explain endocytosis like I was a 5 year old and read it to him... he says; "Ask it to explain it like a scientist now." After that he said it was a really good and we started asking it all kinds of biology questions.

I happen to agree that search will be the first thing disrupted. However, I think simply saying "search" doesn't come close to capturing how deep this will change the way we think, use and progress in terms of the way we define "search" right now.

I've got a young kid and I'd think twice before letting this model explain any science to him. If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you.

I think you have a point about your tkinter example. That kind of stuff _is_ a lot more convenient than googling and copying and pasting code. But if you push it beyond stuff that you could easily find on stack exchange or in documentation somewhere it doesn't work that well. Like I said, its a search engine with a lot of downsides and some upsides.

> If your criteria for whether a model is good is "it fooled a 9th grader" well, I don't know what to tell you.

Fooling a 9th grader is amazing. That's a pretty well formed human being right there except with less life experience. Fundamentally no different from you in general reasoning terms except on a smaller set of information. So fooling you is merely a question of model size.

"Fool" is the operative word here. ChatGPT is quite capable of producing very plausible sounding text about biology that is totally incorrect. See, for example, the example in my comment at https://www.lesswrong.com/posts/28XBkxauWQAMZeXiF/?commentId...

You're basically complaining that a single model doesn't have full knowledge of every single area of all of human knowledge. It's got decent knowledge of most areas including programming with probably better overall knowledge than a high school student. That's downright amazing and probably more knowledge than any single human actually has. The rest is likely a matter of improvement along the same lines versus some radical redesign.

Well, I agree that it's amazing - it almost always produces grammatical output, for instance. But it's not a reliable way of obtaining knowledge. One should not, in particular, try to learn about biology by asking ChatGPT questions. It often produces made-up stuff that is just wrong. And it's very confidently wrong, with the output often coming across like someone barely concealing their contempt that you might doubt them.

It may or may not be fixable without radical redesign. The underlying training objective of mimicking what humans might say may be too at variance with an objective of producing true statements.

My wife (a physician) asked it multiple medical questions and the majority of the time they were dangerously wrong, but looked perfectly fine to me.

I asked it a series of questions about my area of expertise and they were wrong but looked perfectly fine to my wife.

It even confidently “solved” the 2 generals problem with a solution that looks completely plausible if you don’t already know that it won’t work.

This was my experience when I asked it questions about database or distributed systems. I wouldn't consider my an expert in these fields, but I do have more than a passing interest in them.

All the answers looked good, used several of the correct terms, and one even referenced the project I worked on, but they just contained flat out wrong information.

You reach a point, where when you ask a ML model to generate text given the internet as a corpus, where there just isn't enough text to make something that is both true and convincing. In niche fields, this is just where we are at.

Do my core work? No, it's not going to, at the moment.

But it's already saving me nontrivial amounts of time on tasks like "write a polite followup email reminding person X, who didn't reply to the email I sent last week, that the deadline for doing Y expires at date Z".

I typically spend at least 3-4 minutes finding the words for such a trivial email and thinking how to write it best, e.g. trying to make the other person react without coming across as annoying, etc. (Being a non-native English speaker who communicates mostly in English at work may be a factor). ChatGPT is really good with words. Using it, it takes a few seconds and I can use the output with only trivial edits.

Edit: I also see that I am falling prey to exactly what the paper itself is talking about.

"The more adept LLMs become at mimicking human language, the more vulnerable we become to anthropomorphism, to seeing the systems in which they are embedded as more human-like than they really are. This trend is amplified by the natural tendency to use philosophically loaded terms, such as "knows", "believes", and "thinks", when describing these systems."

--

An ignorant statement / question I have is why are you using it write code? It's a chatbot, no?

As you've mentioned, it's a really powerful search, and is like having a conversation with someone who is literally the internet.

For example "What is the glycemic index of oatmeal?"

"What is Eihei Dogen's opinion of the Self and how does it differ from Bassui's?"

I get highly detailed and accurate output with these.

The first question is simple and the second is far from it. It's breaking down two Zen masters experiences and comparing them in an amazing way.

I've been thoroughly impressed with Chat GPT so far.

Ask it to breakdown the high level points of a book you've read.

Ask it to rewrite a song in the style of a different artist.

It's so cool, I feel like I legitimately have an answer to any random question at my finger tips and have to do zero filtering for it.

"An ignorant statement / question I have is why are you using it write code? It's a chatbot, no?

I've found it so incredibly useful to simply replace Google."

Heard of Stack Exchange?

I teach and I expect many students to use language models like ChatGPT to do their homework, which involves writing code. Lots of what people are doing with it is coding (there have been quite a few posts here using it that way).

I've actually also used ChatGPT for literary/song writing experiments and it stinks, aesthetically. The lyrics it wrote, even with a lot of prompting, were totally asinine. And how could they not be?

I used chatGPT to solve a sqlite bug involving a query that was taking 4 seconds to run. I pasted the query and it identified many possible issues with the query including the offending problem (it was missing an index on a timestamp).

It also passed 3/4 of our companies interview process including forging a resume that passed the recruiter filter.

That being said, I COMPLETELY agree with you that chatGPT will not disrupt anything. Your example cases are completely as VALID as are my example cases.

chatGPT is, however, the precursor to the thing that will disrupt everything.

Points taken, but LLMs are still outpacing expert predictions, so empirically they’re under-hyped.

LLMs are polarizing: while there is a lot of hype from some quarters, there's also a faction that seems only interested in dismissing them as the same old something-or-other.

What I personally find most interesting about them is what seems to me to be their unreasonable effectiveness, despite their flaws and limitations, and what that might tell us about ourselves. The more one stresses how simple (conceptually) their method of operation is, the more surprising their capabilities seem - to the point where I wonder how much of everyday human dialogue is being produced this way.

This vein of innovation may start showing diminishing returns at any time, but if it keeps going for a while, It might deliver insights into what human intelligence is.

Well, my take here is that LLMs are do about half of what makes human intelligence work well: which is recall tons of information using relatively broad and imprecise queries or reminders. And even this they don't really do: humans are quite capable of introspecting about what they know and this seems to be a challenge for this class of model. A challenge I don't see this architecture really easily addressing since, in the end, its just a big markov model with chunking.

But anyway, the other part of the equation is missing here. When a human encounters a novel phenomenon about which they know little they can engage an entire separate system: one which _reasons_ about the system, uses principals and intuitions and iteration to produce new knowledge. That is the thing missing from LLMs: they don't ever really produce new knowledge, although they may reveal correlations between texts that people have yet to notice.

I'm not fundamentally skeptical about whether artificial intelligence will ever get there. In fact, the progress of LLMs has me wondering if its not going to be sooner rather than later. But at this moment, I feel quite confident saying that LLMs are just knowledge retrieval systems with some pretty undesirable properties (and some pretty interesting ones).

My _hunch_ is that the next step this is going to be like self driving cars, though: technology which appears stubbornly _just out of reach_ for an indeterminate amount of time.

> This will hardly seem like a controversial opinion, but LLM are overhyped. Its certainly impressive to see the things people do with them, but they seem pretty cherry-picked to me. When I sat down with ChatGPT for a day to see if it could help me with literally any project I'm currently actually interested in doing it mostly failed or took so much prompting and fiddling that I'd rather have just written the code or done the reading myself.

> You have to be very credulous to think for even a second that anything like a human or even animal mentation is going on with these models unless your interaction with them is anything but glancing.

I've used ChatGPT, and I'd say it's right now as useful as a google search, which is already a lot. Most humans would be absolutely unable to help me (and probably you) for your projects because they aren't specialized in that area. That's not even talking about animals. I love my cats but they've never really helped me when programming.

Maybe I’m just old, but there just isn’t much that I want to computers to tell me about that they don’t already do a decent job at. Everyone loves to complain about how bad Google search is, but I very rarely find myself desperately looking for something and unable to find it. There’s certainly no normal conversational interactions I can think of that I would love to have with a computer but have been unable to before ChatGPT and similar.

That limits how impressed I can be by ChatGPT and similar beyond just being impressed by it on a purely technical level. And it’s certainly very technically impressive, but not in some transcendental way. It’s also very impressive how could recent video games with ray tracing look, or how good computers are at chess, or how many really cool databases there are these days, or how fast computers can sort data.

The difference in wether you think ChatGPT is game changing or another overhyped LLM seems to come down to:

1) do you acknowledge prompt engineering is a real skill set?

2) are you willing to improve your prompt engineering skill set through research and iteration?

There is much to learn about prompt engineering from that “Linux VM in ChatGPT” post and other impressive examples (where the goal of is to constrain ChatGPT to only engage in a specific task)

It is very, very good with language, and very bad with facts and numbers. That's an oversimplification, but also the gist of it.

You have to recognize how it works, why it works - then you can use it as basically an incredible superpower force multiplier.

Understanding the lifetime/scope of variables is not easy, many experienced programmers get it wrong, especially when moving between languages.

I hope ChatGPT in its current form will not be used for search. As my friend says it, ChatGPT is not intelligent, it’s just capable of creating responses like it’s knows everything. The things it hallucinates is likely going to spread misinformation and make it harder for the masses to search for true, factual information.

The other part is webtraffic: Google in theory could have created an interactive, conversational style search engine (with it without LLMs) if they wanted to, but a lot of websites would have complained about Google taking away traffic from them. I believe the same happened when Google started showing it’s own reviews instead of redirecting to Yelp. I wonder how openAI or any LLM powered search is going to deal with it. They don’t have to worry about it anytime soon, they still have a lot of time to get to a stage where they come anywhere close to the number of queries Google handles in a day, but it’ll be interesting to see how things go.

The problem of hallucination in LLMs is a well-known and studied problem and solutions have been proposed to counter it. The most promising one is augmenting LLMs with a retrieval system. This involves sourcing a large database of factual information, say journal articles, over which the LLM uses an information retrieval system(search engine) to extract information on which its generated output is conditioned. Recent job postings from OpenAI suggest that’s their next step of development for these LLMs.

I think critics of these LLMs are missing the point about the excitement around them. People are excited because of the rate of progress/improvement from just two years or a year ago. These systems have come a long way, and if you extrapolate that progress into the future, I predict majority of these shortcomings getting resolved

Thing is, ChatGPT is already incredibly useful for searching random things you know enough about you can evaluate responses critically. The alternative here is doing a regular search, and wading through SEO-bloated, ad-laden content marketing "articles". The quality and reliability of information is about the same (or even favoring ChatGPT), but without 90% of the text that's just filler, without bullshit, ads, upsells, tracking scripts, etc. I tried it a few times and it's a much better experience than the web. I'm gonna be using it for as long as it lasts.

Yeah, but its not as reliable as just restricting your search to Wikipedia or the appropriate academic journals or even chatting with a librarian!

Sure, when the topic matters or I need to study it in depth, I can still go to Wikipedia or PubMed or Arxiv.

But there are plenty of searches one does that are trivial, or serve to illuminate the problem space, and cover topics that in which I can rely on common sense to correct wrong advice. And the issue with non-technical topics, the kind applicable to mass audience - like e.g. cooking or parenting or hygiene - are very hard to search about online, because all results are bullshit pseudo articles written to drive traffic and deliver ads. So it's not that ChatGPT is so good, but more that Internet for normal people is complete trash, and ChatGPT nicely cuts straight through it.

Indeed. If I could have the Google from 20 years ago, I probably wouldn't be so impressed with ChatGPT as search engine.

But with the Google (and the web) of today, where it's practically impossible to find reliable information about many subjects without adding "site:reddit.com" or "wikipedia", I find it extremely useful.

But if so this isn't because of its nature (the fact that it's an LLM), but because of its inputs. An LLM fed the same bullshit pseudo articles you refer to would likewise spit out more bullshit. If ChatGPT works it's because its sources have been carefully curated.

Fair. But the practical reality right now is that ChatGPT delivers useful results without the noise, whereas normal web search does not. It blows the web out of the water when it comes to value to effort ratio of generic web searches. It won't last forever, but I'm enjoying it for as long as I can.

I agree that I'd still rather use a search engine over a small set of sites than ChatGPT for exactly the reasons you suggest and others. But I don't see ChatGPT as having a lot of utility beyond functioning as a search interface for credulous dummies. I mean if I were literally developing a chatbot then clearly its a pretty interesting technology (assuming its problems can be tamed or censored somehow), but beyond that I don't really get it.

The problem you are running into is that you are overindexing on the fact that LLMs will sometimes be wrong and you are used to using technology that is basically always right. But we are in the early stages of LLM adoption - correctness will improve (see for example citation driven LLM-search) but more importantly, the set of LLM-driven applications that can be probabilistically correct and still wildly useful will grow.

LLMs like ChatGPT are just so damn cheap for the power they provide, it's inevitable