Maybe we have different definitions of inputs and outputs, but all of those things seem like outputs to me? Maybe in this scenario the inputs and outputs are really the same set of variables so there isn't a huge distinction.

The reason I linked that article is that all the inputs are outputs of the network, so by definition it's training on just outputs (under my personal definition of outputs).

I specified what I meant by inputs and outputs in my original comment. Whether it rained today, whether it was cloudy yesterday, how hot is it going to get, that sort of thing, are the observable weather, the things most people care about when checking a weather forecast. But you can't only train on those things and expect to come up with accurate predictions. You also need to train on the things that cause that observable weather (air currents, warm fronts, etc.)

It just doesn't know how to compose knowledge. It knows the letters in "blueberry" if you ask it, and it knows how to identify the position of a letter in a sequence. But it doesn't know how to get the letter R in blueberry since the composition of the above two actions isn't in its training set, hence it reliably fails such questions.

That proves in general these LLMs can't compose knowledge, unless it has seem a lot of examples of similar compositions before. That is a massive problem and why I believe LLMs will never reach AGI, you need something more.

> But it doesn't know how to get the letter R in blueberry since the composition of the above two actions isn't in its training set, hence it reliably fails such questions.

Me

what is the seventh letter in the word "blueberry"?

ChatGPT

The seventh letter in the word "blueberry" is "r".