Show HN: 1v1 coding game that LLMs struggle with
1v1 strategy game I have been building for a while as a side project. It's purely a passion thing that has no aspirations for being anyhow monetized, though I hope to make it enjoyable to play.
I let LLMs play a mini-tournament. Here are all the replays and results of their games: https://yare.io/ai-arena
All are able to produce 'functioning' bots, but they are nowhere near even weak human-coded bots, yet
(
8 comments
| 3 points
| on HN
)
A question though, why such powerful bots like Gemini 3.1 failed against Clowder bot? Is it because of inefficient code or the LLMs did not handle edge cases? Or they are not as good as humans when it comes to strategy.
You can see replays for all of the matches if you hover over the cells in the table.
Show HN: Yare 2 – Programmable RTS game - https://news.ycombinator.com/item?id=32394902 - Aug 2022 (26 comments)
Show HN: Yare.io – game where you control units with JavaScript - https://news.ycombinator.com/item?id=27365961 - June 2021 (64 comments)
(Btw, reposts are fine after a year or so; links to past threads are just to satisfy extra-curious readers!)
From the prompt it looks like you don’t give the llms a harness to step through games or simulate - is that correct? If so I’d suggest it’s not a level playing field vs human written bots - if the humans are allowed to watch some games that is.
I did try letting the models iterate on the bot code based on a summary of an end-of-game ‘report’, but that showed only marginal improvements vs. zero-shot
Step(n) - up to n steps forward
RunTil(movement|death|??) - iterate until something happens
Board(n) - board at end of step n
BoardAscii(n) - ascii rep of same
Log(m,n) - log of what happened between step m and n
Probably all this could be accomplished with a state structure and a rendering helper.
Do you let humans review opposing team’s code?