timomo

I dont agree with most of the comments in here. I also consider the human mind a gigantic pattern matching loop and do not consider me a laymen...

I think Transformer models (like chatGPT) can encode knowledge of the world into their representations as well as work with the encoded world knowledge when predicting. Consider the example of the apple that falls: I am sure the embedding (internal representation of words in ChatGPT) for apple contains some form of "physical objectness" that will distinguish it from a word like "vacation". It can also put this "physical objectness" into context and infer what happens and what cannot happen when you let it the apple go on earth vs in outer space. Maybe it would be good for the sceptics to try ChatGPT and ask "What happens to X when you let it go from your hand on earth/in outerspace? please explain your reasoning." And fill in X with any object or concept that you can think of.

To add here: for a local minimum to occur all those dimensions (or features) need to increase. This is highly unlikely for modern NNs where you have millions of dimensions. If one of the dimensions is going down but the rest up, you have a saddle point. Since you go down only one (or few) dimensions it takes longer.

What is the population of Cambridge?

for me the demo returns that the model did not find an answer...

btw the demo can be found at https://haystack-demo.deepset.ai/

First come first serve : )