These models have limited agency for the moment —- they just answer questions. But I see no reason it couldn’t improve itself in principle if it was given agency enough to do so.

The models embedded in Bing and ChatGPT have limited agency but through the APIs people have given them all sorts of agency (eg using https://langchain.readthedocs.io/en/latest/modules/agents/to...). I'd be extremely surprised if there aren't already self improving bots out there.

Check if you replace the “ValValVal” formatting you mentioned with spaces (such that “XOO” becomes “X O O”). The reason I suggest this is that the NLP tokenizer would set “XOO” to 1 token, which could cause interpretation issues.

Very interesting... I tried your suggestion and fixed a few other typos / inconsistencies in my prompt.

Here is the new version: https://pastebin.com/BSUGtxfb

It seems to be performing better! It played 2 games without mistakes, and was able to evaluate diagonals correctly. I just hit my usage limit for GPT-4, I'll do more testing later.

I asked GPT-4 to give me a prompt to use with it in order to set up a game of tic tac toe and it came up with a similar representation and played successfully.

https://pastebin.com/83iZjHuc

I'm still blown away that this is possible.

This worked for me with GPT-4:

    Let's play tic tac toe.  After every move please check the game state to see if anyone has won.  You go first.

I just tried with gpt-4 and it failed on the first attempt :(

https://pastebin.com/gSm0bbH9

It didn’t realize that it had won.

I would imagine that its training data inclucded millions of chess games and not many tic-tac-toe games.

I would imagine it has both. And since it’s much easier to play legal moves in tic-tac-toe (just find an empty square) it should be much easier. In fact there are less than 20k total tic-tac-toe games so unlike chess you could just memorize every game.

The fact that it can’t even do that tells us that it doesn’t have an internal board representation to reference.

OTOH if the training data includes just 1000 tic tax toe games, that's potentially enough for every possible outcome

How does it do on "Let's play connect four." ?

Well, it pretended to have won but failed to see the 3 X he placed were actually not vertical :-(

https://pastebin.com/LXLhXZjf

Hmm, I used a much simpler prompt and actually got to play a valid game with it. My prompt was:

> Let's play tic tac toe! We will take turns naming our moves and each time you move please list all moves so far. If either of us wins the game just acknowledge it and print the state of the game. If I win just acknowledge that and end the game.

> I start with the center spot.

It wasn’t particularly good at the game (missed opportunity to block me from winning) but it played valid moves and detected when I won: https://ibb.co/L83JHxP

Here's what I tried with Tic Tac Toe. There's one error in its first response but otherwise it follows along seamlessly. I was really surprised. I'm interested if you can get the same results.

https://pastebin.com/c833TSV2

(It leaves out the first row of the game in its first response, but I was able to interpret the board state since it played on the second row. When I gave it the full board again in my answer it picked it right up.)