Instruct GTP4 to win the game and then crush it using Stockfish. Does it handle defeat gracefully?
It sounds like -4 should be much better though; I wonder if having the gameboard would improve its gameplay?
If it's an addition, then sure seems like it wouldn't hurt. If it's to replace the algebraic notation, then I'd say very likely worse. All of it's input / experiences w.r.t. chess have been in the language of algebraic chess notation. Swapping to only providing a game board would presumably be worse.
It'd be like if all of our interactions were in English in a latin script, and then someone replaced it with emojis. We don't think in emoji's it wouldn't help us, even if that's how they're used to thinking.
While we play chess by looking at the board state, it's likely little of chat-gpts input is in that format.
And the only other concern would be memory. Would it "forget" the prior moves. But I think not, simply because it receives the full game input each play simply by how GPT APIs work with passing in all prior tokens after each iteration.
Of course, some humans can play chess quite well while blindfolded.
There is a waiting list https://openai.com/waitlist/gpt-4-api
I do hope the chat interface improves soon. It seems like low hanging fruit compared to everything else that's been accomplished ... it will be like the jump from telnet/irc/ftp to web browsers.
I enjoy the ChatGPT jailbreaking scene, and almost every time there's a significant break found, within a day, ChatGPT will dance around it (take the big token-leak jailbreak from a few days ago.) The GPT4 model may not be retrained itself, but it seems fairly evident that the ability for supplemental rules/fine-tuning is baked into their architecture for ChatGPT.
[1] https://langchain.readthedocs.io/en/latest/modules/agents.ht...
It doesn't play like stockfish, but it does seem to play pretty well.
If you specify a move or capture that is nonsensical it seems to always recognize that
You can stop and ask it about the tactical reasoning behind its moves, and it gives what look (to me) like good, specific answers
You can ask it if its concerned about an attack or defense I might make, and it will give a good answer. It will even answer accurately if the attack/defense is not possible
You can ask how the game is going (who is likely to win) and it will give an accurate answer with good reasoning.
You can ask it to try to meet specific alternative goals, like opening itself up to an en passant attack, and it will attempt to accomplish those goals. This one is really astonding since you're asking it to play in a way thats strange, and which is probably underrepresented in its training set.