This is a black swan event which, if anything, vindicates the hype from dreamers who were ahead of their time. They had a lot of the theory right but lacked the horsepower. This will complete the last mile for other AI tech and be the lacquer that gives it the all important finishing touch.

"This is a black swan event"

By definition, black swan events are unpredictable. This example isn't that.

LLM emergence was unpredictable in 2017. Even for the best AI people.

ChatGPT is nothing more than long-form autocomplete. The specifics of approach are irrelevant.

A human brain is nothing more than long-form autocomplete. The specifics of approach are irrelevant.

Exactly. All the people saying LLM's aren't important because it is just auto-complete, really don't connect the dots that humans are also just auto-complete.

What on earth are you talking about?

Our brain excels at pattern matching. We attribute a lot of human abilities to “intelligence”, but the line between that and pattern-matching (or “autocomplete” ad it’s being referred as here) is being challenged by these latest LLMs.

If it can look at a picture, explain what’s in it, and hypothesize about physics inside the picture’s environment, is it “just pattern matching”?

"If it can look at a picture, explain what’s in it, and hypothesize about physics inside the picture’s environment, is it “just pattern matching”?"

That's extrapolation via approximation. Computers synthesize a specification. The difference is nuanced and entirely contextual.

Those are way too broad terms for this conversation to go anywhere. The point is, what's to say the LLM is not doing "extrapolation via approximation" or some process that is analog to how we think? We barely understand how the brain works to start with.

But the human brain is a much more mature product (although I will concede there are still a ton of bugs to work out that we really have no idea how to repair because the product is extremely complex and still pretty fragile).

Everyone likes to say this, but no one seems to provide evidence. I don't believe it.

Why do you think that?

I knew something changed after AlphaGo. Compute could do what we thought only true intelligence can do. So I agree, LLMs are not a black swan. It will change everything, nevertheless.

I keep reading these opinions that LLMs are just doing some advanced form of copy paste. Actually, we don't know what they are doing. Are they actually doing some form of modelling and abstraction? Seems likely to me.

"I keep reading these opinions that LLMs are just doing some advanced form of copy paste. Actually, we don't know what they are doing. Are they actually doing some form of modelling and abstraction? Seems likely to me."

This is exactly the problem with AI. For business or government, the answer is as important as the methodology employed. A black box does not work for the majority of use cases.

Until it can show its work, it's a sideshow.

Humans are also "black boxes" we don't fully understand. It's worked pretty well.

This is a shallow counterargument that entirely misses the point.

This is not even an argument, only a disagreement.

"This is not even an argument, only a disagreement."

It's a straw man.

The need for transparency in process is known, documented, and undisputed. Your comment has no relevance. My brain might be a black box, but I can still communicate and/or document the specifics of a process.

Can [insert your preferred model] do that? Didn't think so.

Given the changes in health over the past century and a half as we've adopted "science" into food, I'm not sure that I agree.

Yes, in many situations it is important that we get AIs to explain how they came up with a result, and that we are confident in the correctness of the explanation. That is probably the most interesting thing you can work on now. Is that a sideshow? I don't think so, although I wouldn't mind that at all.

AlphaGo is basically the same as image recognition, not sure why you thought AlphaGo was special compared to detecting faces. AlphaGo works the same way but maps game structures on a Go board instead of mapping facial structures and comparing those to known people.

It was able to beat the top GO player. What does it matter if that technology is also used in image recognition. I'd say that if its 'brain' can play games and recognize people, or in case of ChatGPT4, can talk and play chess, this is all starting to sound pretty 'General' to me.

That's exactly the kind of attitude I am talking about. You can rationalise pretty much everything and keep moving the goal post. I've seen AI do something nobody thought was possible. If that doesn't influence how you think about AI I cannot help you.

Nobody?

Well, maybe the self-driving car people thought it was possible. But I don't take them seriously.

next bit prediction, or long-form autocomplete, may lead to general intelligence. http://mattmahoney.net/dc/rationale.html

The black swan event includes the unexpected emergent capabilities of LLMs, which can pass professional examinations and exhibit many attributes of general intelligence, combined with the broad availability to market and massive pressure on big industry players to adapt and innovate or die.

If things keep going this way then pretty soon the black swans are going to outnumber the white ones.

I don’t put much stock in the claims about GPT4 “passing” professional exams. Many copies of previously administered exams are available, and the exams are formulaic in their construction (to make them stable, predictable targets).

A compressive copy of the internet brute-forcing its way through an exam (which it may even have digested already) is really not interpretable as performing well on the exam. It’s a meaningless measure because the tests were not designed with this use in mind.

Single stories of LLM success are just as problematic as single stories of LLM failures. As always. The fact is that we have a dangerous tool at hand that absolutely REQUIRES skepticism if you are trying to get *facts* out of it.

I agree that LLMs are extremely likely to impact many areas of work, particularly bullshit work. But as it stands you absolutely cannot use them as fact machines, the results can be catastrophic.

What it does well, among others:

- Scaffolding text, breaking writers block etc.

- Compose basic texts from minimal input, for example for bullshit tasks -> I generated an internal "vision statement" during a Miro workshop for my team by inputting a bunch of bullet points gathered from the team members brain storming. It created a concise, fluid text that everybody liked. It's now the vision statement.

- Point you in good directions, give you ideas

What it does NOT well, among others:

- provide factual responses. all responses MUST be scrutinized because they are likely containing false information. This is very dangerous for society ("Can I take this medicine with this other medicine?")

- Compose creative texts that are coherent and novel. ChatGPT texts can be quite fun but they rarely make sense beyond very superficial screening and convey no deeper message.

However, ChatGPT-like tools are used with a lot of naivety and often blind acceptance instead of using them as tools to aid your work.

For the most part I agree, but will qualify that the caveats you listed apply to the out of the box version of ChatGPT. I expect these limitations will be overcome by using it as programming substrate and connecting it to other models and APIs.

I am impressed by what we’ve seen from ChatGPT so far, but am especially excited to see what industry does with LLMs as new type of building block.

Fully agreed! I suspect that there will always be this subtle risk of catastrophic failure though, something observed in a lot of AI systems. The scrutiny filter may become less relevant but will likely not be less needed to prevent 1 in 10,000 bad responses.

If 100,000 people ask critical questions then 10 people might run into potentially catastrophic consequences. ChatGPT is a powerful tool and will only become more so but it will probably not be perfectly reliable by any means due to the nature of the system.

I am excited for the generative AI future and whatever the hell is still coming. Only those who adapt will survive.