Re:Write Code, Crush Foes, Get Hired – Multiplayer Programmi...
by michaelmior on Tuesday May 20, 2014 @02:20PM
(#7774223)
Thanks for the response! So in theory, a player could game the public leaderboards by returning incorrect results for a simulation? This seems like a reasonable choice.
Re:Write Code, Crush Foes, Get Hired – Multiplayer Programmi...
by schmatz on Tuesday May 20, 2014 @02:28PM
(#7774265)
Yep, they could.
If people start doing that to mess up results, we have ways of remedying that situation. We haven't had to deal with any cheaters thus far as our users are pretty awesome (and there really is no reason to cheat).
We'd probably send each game out multiple times and require consensus or run all the simulations ourselves. I don't anticipate having to do anything of that nature, but it's interesting problem to think about!
Those simulations are used to calculate the public leaderboards during the competition. They aren't used for final rankings as the simulation results need to be validated. However, they provide a nice way for people to gauge their standing.
To perform the final ranking/validation, we'll play n of the top games on each team against each other and then sort by wins and losses. We're not quite sure of how large n will be, but I think it will be 150-250. It would be nice to calculate the public leaderboards this way, but as a O(n*m) calculation, it really doesn't scale in practice (we have to strike a balance between accuracy and responsiveness of the ranking system). The public leaderboards are calculated with a Bayesian algorithm.
If you have any other questions, feel free to ask! :)