I just tried with gpt-4 and it failed on the first attempt :(

https://pastebin.com/gSm0bbH9

It didn’t realize that it had won.

I would imagine that its training data inclucded millions of chess games and not many tic-tac-toe games.

I would imagine it has both. And since it’s much easier to play legal moves in tic-tac-toe (just find an empty square) it should be much easier. In fact there are less than 20k total tic-tac-toe games so unlike chess you could just memorize every game.

The fact that it can’t even do that tells us that it doesn’t have an internal board representation to reference.

OTOH if the training data includes just 1000 tic tax toe games, that's potentially enough for every possible outcome

How does it do on "Let's play connect four." ?

Well, it pretended to have won but failed to see the 3 X he placed were actually not vertical :-(

https://pastebin.com/LXLhXZjf