News for hackers. Stuff that matters.
Re:Poker Tournament for LLMs by energy123 on Tuesday October 28, 2025 @05:21AM (#45730723)
Not enough samples to overcome variance. Only 714 hands played for Meta LLAMA 4. Noise in a dashboard.
Re:Poker Tournament for LLMs by mpavlov on Tuesday October 28, 2025 @07:49AM (#45731619)
(author of PokerBattle here)

That’s true. The original goal was to see which model performs statistically better than the others, but I quickly realized that would be neither practical nor particularly entertaining.

A proper benchmark would require things like: - Tens of thousands of hands played - Strict heads-up format (only two models compared at a time) - Each hand played twice with positions swapped

The current setup is mainly useful for observing common reasoning failure modes and how often they occur.