Re:Show HN: A Word Guessing game like Codenames that allows ...
by cheekygeeky on Friday November 06, 2026 @04:52PM
(#50399002)
I had one other thought: The LABELs given to the models assume that the models behind the name never change (GPT 6.1 Sol, for instance). I doubt that is true. I just don't think that they update the number behind the name until they have a feature or benchmark that they can report is significantly better. Thus, I think it would useful to reset the model scores to the starting score each day and TRACK/CHART the ELOs of the models over consecutive days. That would test my theory, if nothing else. Does a model "get dumber" as some people feel/report sometimes? Does a model get incrementally "smarter"?
[ Parent
| on HN ]
Re:Show HN: A Word Guessing game like Codenames that allows ...
by daluan on Friday November 06, 2026 @05:01PM
(#50399088)
Some models here are called by their IDs that date them on Amazon Bedrock, for example, the Haiku 4.5 ID is 20251001. Other models sometimes go through a provider's live endpoint, and those can change behind the same name. So drift is possible for some models and unlikely for others. A daily reset would be too noisy and costly (I'm still in school lol). Most models play only a handful of games a day, and Codenames has a lot of board luck through RNG. Potentially, maybe a weekly recalibration is an interesting research idea though.
Thank you, cheekygeeky, for your numerous interesting questions. I'll try to ponder them to improve my site. If you're interested in testing the models for yourself, you can click on the link in my title or visit wordspies.com to get a feel for how different models perform when you play with and against them!