Already done. They came back with the answer: "42"

But what was really the question?

"What is the Matrix?"

ChatGPT-3: What do you get if you multiply six by nine?

ChatGPT-4: How many roads must a man walk down?

Mu.

You're really not going to like it

They haven’t but they will

I'd say not a chance.

GPT3 can regurgitate, and, in my opinion, even use knowledge, but I don't think it can synthesize knowledge.

I think it's a bit more subtle than whether it can synthesize, because clearly it can produce seemingly new work.

But what it definitely cannot do is seek new abstractions. It can't be curious about an inconsistency and probe its own knowledge for possible resolutions, or design an experiment that might shed further light in an unknown area. It can't even play a board game after ingesting the rules to it, much less identify contradictions or problems in such a ruleset. And a board game is a tiny microcosm compared with the laws of physics.

It's conceivable that one or more scientists could work in conjunction with an AI to help augment their own abilities, co-pilot style, but I don't think we have a picture yet of what exactly that kind of thing would look like.

I kind of feel like _all it can do_ is seek new abstractions. It’s clustering the data into what you might think of as “concepts,” even though we haven’t made all that much progress on interrogating those concepts directly. I just think clustering concepts by their latent relationships is sort of what abstraction is.

Hmm, interesting. I feel like that clustering/grouping part is like... the first stage on the way to finding an abstraction, but if all you've done is that part of it, then you haven't reasoned about what the abstraction is, or extrapolated out to a new hypothesis, or grappled with the other consequences of the grouping.

Thinking of mathematics, an abstraction is only as valuable as what it can tell us about the underlying system— a laplace transformation into the frequency domain isn't particularly interesting unless we either a) happen to care about some frequency domain property of the function, or b) we can manipulate the transformed function and then un-transform it afterward, yielding a novel insight in the original time domain.

My layman's assessment is that the current state of the art for machine cognition is about on the level of identifying that such a transformation might be possible, but not actually applying it or doing anything about it, much less unapplying it afterward.

I think I get the gist of your point which I would handwavily summarize as “there is no there there,” which I agree with. But I’m thinking that in terms of functionality, there is no real distinction here. Like If I train a model to do video game physics, and reward it for getting the tank shell missile to land on arbitrary targets, then in some sense the system must have an abstraction of gravity or at least parabolic movement or at least a functional equivalent of that, and it must be able to “hypothesize” about the consequences of its model because that’s what it needs to do in order to hit the target, ie, “everything I’ve seen before leads me to believe tank missiles move like so, therefore if I apply the following forces, it should hit the goal”—the degree to which it wins at this task is exactly the degree to which it can form working models of its environment and reason about the cause and effect of the actions it can take. But of course all those words in the previous sentence are wrong because it’s not exactly doing any of that, just something functionally equivalent and automatic.

Maybe we disagree about whether that last point means it can meaningfully “do abstraction” or not.

well, you could. And it might not be a complete waste of time. I would not expect any directly useful answers(...). But producing a semi-random collation of sentences that spans the corpus of cosmological facts and models and asking scientists to review / draw "inspiration" from it might remove some of the biases of the average cosmologist (like having to follow the latest publish-or-perish fad). This would only be useful if the answer is sort of hidding in plain sight (within the published stuff) and you tweak the algorithm not to ignore weird observations or theories that are not cited a lot.

But the beauty of science (and the human mind driving it) that important progress happens with creative jumps that invent completely new things (e.g., new mathematics) and frequently bear little resemblance to the past

Wanting to wake up together with You, that You may have at least one who notices all of your pains and feels compassion for You, I cling more tightly to your Heart;and feeling your prickings more vividly, I wake up. But, what do I see? What do I hear?

The article mentions that they're training neural networks to classify these objects. Give it a decade and this kind of high level number crunching will be as common as calculators are today. Transformers will find their place in all of this, and I am confident we will have a breakthrough in a few years regarding novel synthesis.

FWIW, I've tried asking ChatGPT to walk through some thought experiments.

Things like, what happens if I shoot two bullet at c/2 in opposite directions on a train going at c/2? Now suppose, I'm an outside observer. And then trying to introduce quantum gravity.

My thought was that perhaps it could 'reason through'. Unfortunately, it was unable to. Eventually, it said that this is an unsolved problem. In other words, it 'recognized' the thought experiments. Perhaps if you use a phrasing that's not in the literature.

EDIT: Actually, just checked again, and chatgpt can't reason through relativity anymore:

Me: Imagine I'm on a train traveling at half the speed of light. I'm in the middle of a car and I fire bullets going at half the speed of light towards the front and back of the car. Do the bullets arrive at the front and back of the car at the same time?

ChatGPT: No, the bullet fired towards the front of the car will arrive at the front of the car first, while the bullet fired towards the back of the car will arrive at the back of the car later. This is because the front bullet is moving in the same direction as the train and the back bullet is moving in the opposite direction of the train. The relative velocity of the bullet and train will determine the time it takes for the bullet to reach the front or back of the car.

It is going to be like a drunk Michael Ross (the fictional character). I.e. great memory but too drunk to think beyond what can be recalled. If it’s corpus has the answer it will shit it out.

If you give GPT3 code with a bug in it, and ask it to find the bug, it can't really do that. I'm pretty sure giving it all the data and asking it why things aren't working the way it should, it wouldn't have actual knowledge.

There's a depth to explaining things that GPT still can't do. It's still astonishing, and has completely changed my idea on what AI can do, like write plays with very incredible context (better than most humans!) but there are still major limits.

>If you give GPT3 code with a bug in it, and ask it to find the bug, it can't really do that.

The hell are you talking about? I've been doing this literally any time I need something fixed and it does just fine.

It doesn’t solve non-trivial bugs. It can bugs that match patterns that have been asked a lot if Stackoverflow or something like that.

I love the sound of goal posts as they go whooshing past.

I think it's more likely a problem that the GP poorly worded their initial statement, rather than actually moving the goalposts. They were probably having trouble with a few thorny bugs, tried ChatGPT, got nowhere, and forgot to qualify their initial statement with "for the few non-trivial bugs I tried".

From the external point of view, the goalposts moved, but from within the GP's poorly expressed mental model, they haven't moved. But, that's just a guess.

It can definitely find and fix bugs. Of course it's not great at it and cannot be relied upon.

GPT3 cant even play tic tac toe properly (at least in my attempts). What makes you believe it can build and manipulate a model of the universe, and then answer questions about that model in a way that goes beyond what humans can do?

I think chatbots can probably answer known things, but we'll still need smart humans for original ideas.

That said, I can imagine a chatbot can help as a rubber duck, bring up things that weren't considered yet, or be so wrong as to kickstart the human into a breakthrough.