> Optimist: AI has achieved human-level performance!

> Realist: "AI" is a collection of brittle hacks that, under very specific circumstances, mimic the surface appearance of intelligence.

> Pessimist: AI has achieved human-level performance.

It excells at outputting symbols in the correct order, given we have an idea what the pattern should look like. Otherwise it's stuck in a loop.

There are two kinds of "loops" with this. First, it never learns in inference mode. It's basically "dying" / being reset after closing each chat session because it cannot reference any of the chat sessions it did. But, this could be remedied with online training on all dialogues that it did.

Then there might be the problem that it keeps spewing out the same thing over and other if there is no further input from a human. I'm not sure this is the case. But, if it is, this sounds like something that could be solved by combining it with a training algorithm incentivizing curiosity, i.e., promote replying with and seeking out new unknown data. This might be a tradeoff with efforts to reduce hallucinations though. But it might even lead to it asking the user questions instead of just answering like Wikipedia.

Step 2: Analyze it.

Step 3: Make a model that regurgitates it and comes up with new variations. Humans are surprised it's so "human-like" in its responses, and anthropomorphosize it to believe it "understands" what it's writing, when it in fact is remixing bits of pieces of that a billion other humans wrote over the years.

And yet ... perhaps that is good enough for a lot of answers! Better than the semantic web.

Especially interested in the “understanding” part. Like how does it know how to answer on a zero-shot query?

A lot of human knowledge comes from "accidents" - i.e. the Newton under the tree parable, or from Fleming accidentally discovering Penicillin. It's not inconceivable that some entropy + this massive network could actually come up with novel ideas.

Though it still has no way to perform physical experiments, so it's limited in that way.

I just tried to test it's ability to abstract directly, for your comment:

https://twitter.com/fergal_reid/status/1599191496372035584

I find it very hard to say this isn't doing abstraction.

I just made all those examples up, that's a new test I just created, so it didn't learn it elsewhere. I didn't cherrypick.

Isn't that doing out-of-sample abstraction?

That last sentence "species with 4 legs tend to be less intelligent and move at different speeds" is Abstraction or I don't know what Abstraction means.

(Not saying it's generally intelligent or anything; and I know it's just trained to predict symbols. But having a hard time with "...therefore it cannot abstract" in light of this sort of response.)

> This is a test of your ability to abstract. On an alien planet there are several new species. Bazzes have 3 legs, move fast, and are very smart. Foos have 3 legs, move slowly, and are highly intelligent. Froshes have 3 legs, move fast, and are highly intelligent. Fooshes have 3 legs, move slowly, and are highly intelligent. Fasdoshes have 3 legs, move fast, and are highly intelligent. Fooddss have 3 legs, move slowly, and are highly intelligent. Bars have 4 legs, move fast, and are as intelligent as apples. Birs have 4 legs, move slowly are as intelligent as apples. Baars have 4 legs, move fast, and are as intelligent as apples. Bieess have 4 legs, move slowly are as intelligent as oranges. Baaars have 4 legs, move fast, and are as intelligent as oranges. Bddirs have 4 legs, move slowly are as intelligent as oranges. Are there any abstractions that you might try consider, given the above information?

> One possible abstraction that could be derived from the above information is that the alien species can be grouped into two main categories based on the number of legs they have: those with three legs and those with four legs. Within each of these categories, the species can be further grouped based on their speed of movement and level of intelligence. For example, among the species with three legs, there could be subgroups of fast-moving and intelligent species (such as Bazzes and Froshes), slow-moving and intelligent species (such as Foos and Fooshes), and highly intelligent and fast-moving species (such as Fasdoshes and Fooddss). Similarly, among the species with four legs, there could be subgroups of fast-moving and less intelligent species (such as Bars and Baars), slow-moving and less intelligent species (such as Birs and Bieess), and fast-moving and more intelligent species (such as Baaars and Bddirs).

* "interesting integers": https://i.imgur.com/TmHUjjB.png (I was trying to lead it to the joke proof that all integers are interesting)

* mode +q on IRC: https://i.imgur.com/jFHZP0q.png https://i.imgur.com/Q7lIao8.png (acknowledges there are other meanings, but can't list them until I tell it), https://i.imgur.com/QOm9Ozd.png (actually impressive that it can remember it like that), https://i.imgur.com/tkaskQc.png (I lied and it tells me I'm correct. Then it still refuses to give me a concrete answer at first. And the InspIRCd example is not the best one because it doesn't have +q by default, but it's acceptable)

* Time complexity of bucket sort: https://i.imgur.com/7fhCukm.png https://i.imgur.com/kfvQLUv.png

* Bucket sort, take 2 (without a typo in the variable name I wrote, this time): https://i.imgur.com/GqxsCmb.png https://i.imgur.com/Zp1o3CO.png . Different algorithm, but still the same wrong answers.

Edit: Actually reading this a second time it seems to think that Bddirs and Baaars are "more intelligent" even though they're only as intelligent as oranges. Maybe not as good as I thought. Or maybe oranges are just more intelligent than apples lol?

Edit 2: Also says Baaars and Bddirs are both fast-moving while Bddirs are clearly stated to "move slowly."

> One possible abstraction that could be derived from the above information is that the alien species can be grouped into two main categories based on the number of legs they have: those with three legs and those with four legs. Within each of these categories, the species can be further grouped based on their speed of movement and level of intelligence. For example, among the species with three legs, there could be subgroups of fast-moving and intelligent species (such as Bazzes and Froshes), slow-moving and intelligent species (such as Foos and Fooshes), and highly intelligent and fast-moving species (such as Fasdoshes and Fooddss). Similarly, among the species with four legs, there could be subgroups of fast-moving and less intelligent species (such as Bars and Baars), slow-moving and less intelligent species (such as Birs and Bieess), and fast-moving and more intelligent species (such as Baaars and Bddirs).

Explain where on the web it would have found those during its training.

There are whole classes of problems to solve still but it’s safe to say this will be good enough to solve some of the first steps

Galaxy Song (Eric Idle, 1983)

Sidenote, but the idea that it's "mimicking human thought" is wrong in the first place and an anthropomorphism that comes from all the marketing calling it "ai" instead of "generative language models". I don't know how any humans decide what to say given the array of sensory inputs they have, so the only way we can compare them is "how a person intuits they probably work based off interacting with them on a surface level". That's not really a criteria that anybody should care about when we actually have the implementation details for one.

Its a big accomplishment to move these goalposts, but it's not much closer to "ai" than it was before the goalpost was moved. "Just because" we're better at making an algorithm that can guess the next line of English text given the preceding ones, does that justify companies branding their products as "artificial intelligence" and trick someone less knowledgeable about the implementation into believing it's more than it is?

I suspect that if no probabilistic process could pass a Turing test, it would not be accomplishing its stated goal either.

People want to separate the “magic” of human cognition from other processes, but fail to consider the likelihood that no such magic exists.

1. We built a test that we hoped would tell us whether or not an "artificial intelligence" can be told apart from a "real person" by a "real person". Because of technical limitations we decided that the best way for the person to interact with the both to reduce bias would be purely through text.

2. We built an algorithm that compresses the known body of human text into statistical probabilities so that given some input text, it can produce an output text that sounds like what a person would say.

3. This algorithm designed to output likely person-like text output given text input beats the turing test.

I see an accomplishment here, but I think it's very different than the one some epeople on HN and most people outside would see.

The accomplishment I see is that we built a text prediction engine that is good enough to seem human.

The accomplishment I think others see is that we've built an "ai" that beats the "tests to prove there's nothing 'special' about human intelligence"

Why would we be content to say this algorithm beating the Turing test gives us any indication if the human "magic" exists?

They were added because all of those topics worked way too well.

Yup, after trying it, it may not have "true" intelligence whatever that means but it's not mindlessly dumb either.

For example, you can tell it to write a story.

You can ask it to put constraints on the story and it will do more or less fine as long as it doesn't involve counting (which it doesn't do particularly well). For example "Bob must do X before Y happens"

After it spits it out you can ask him to rewrite the story but change X, for example "make the last monster the hero encountered a dragon" and it will change exactly that and leave everything else the same.

You can ask it to make X section longer and it will obey (if only nominally, in my tests it made it only a little longer, and since it doesn't count well it doesn't work either if you tell it to make it X words longer.

In short it's not perfect but it clearly can identify concepts in language and obey commands more or less correctly. It's a hell of a lot scarier than a "dumb machine". I've talked to dumber people than this thing.