News for hackers. Stuff that matters.
Re:Automated Testing for League of Legends by finishingmove on Sunday May 01, 2016 @03:19PM (#11607480)
We see the test checks three things:

  Verifies:
    - KogMaw deals less damage to non-lane minions
    - KogMaw deals percentile magic damage
    - KogMaw deals normal damage to lane minions
and the verify method has three assertions. In my opinion, this should be three tests, each with only one assertion.
Re:Automated Testing for League of Legends by d0100 on Sunday May 01, 2016 @05:28PM (#11607934)
They'd soon be drowning in individual tests. And it wouldn't necessarily fit to separate them, considering all those 3 have to be tested on each skill usage. One shot has to obey those 3 rules.

Making each an individual test would triple the test time, pollute the test codebase and not bring any advantage.

Re:Automated Testing for League of Legends by finishingmove on Sunday May 01, 2016 @11:07PM (#11609005)
> Making each an individual test would triple the test time, pollute the test codebase and not bring any advantage.

Would it really? None of what you said here is true.

Re:Automated Testing for League of Legends by quacker on Monday May 02, 2016 @01:17AM (#11609317)
I think it is true. I mean, tripling the number of test cases when you already have 5500 of them is going to slow down your dev process in a significant way.
Re:Automated Testing for League of Legends by chrislloyd on Sunday May 01, 2016 @07:28PM (#11608316)
The advantage in splitting up the assertions is that you get more visibility. If one test fails and not the other two that's extra information you can use to debug you wouldn't have if you have all 3 in one test.
Re:Automated Testing for League of Legends by chii on Sunday May 01, 2016 @09:44PM (#11608744)
If that test fails, the dev will just run it locally with a debugger attached. The cost of extra time in the test for the normal passing case isn't worth it.
Re:Automated Testing for League of Legends by quacker on Monday May 02, 2016 @01:25AM (#11609334)
Right, but consider if all three test cases fail. Then devs (who didn't write those tests) need to determine what those three test cases are and if they're all the same issue or not, which takes time.

Ideally, if just one assert fails, it will produce an error message/logs with enough info to tell what went wrong. Having tests produce great error messages is way more valuable than arbitrary rules about assertion density.

Re:Automated Testing for League of Legends by swanson on Sunday May 01, 2016 @03:34PM (#11607538)
I tend to agree, but sometimes in these kind of instrumented test environments, the setup cost of each test case might be very high so maybe the team found it reasonable to group all the assertions into a single test (especially given that the suite is already taking 1-2 hours to give feedback to developers).
Re:Automated Testing for League of Legends by badloginagain on Sunday May 01, 2016 @03:37PM (#11607551)
It appears they're grouping by ability.
Re:Automated Testing for League of Legends by finishingmove on Sunday May 01, 2016 @04:20PM (#11607694)
I am well aware of that and I agree on being pragmatic, but more often than not, the reason for multiple assertions per test is developer laziness and/or unawareness of the benefits of isolating failures by having just one assertion per test.
Re:Automated Testing for League of Legends by 0x0 on Sunday May 01, 2016 @05:26PM (#11607917)
Seems to me you actually want all three assertions checked in the same run. Otherwise you could get three runs where only the first assertion was true in the first run; only the second assertion succeeding in the second run with the others failing; etc.

I bet the tests aren't pixel-by-pixel frame-by-frame reproducible (they are talking about a one week staging period for tests to prove themselves, so surely there's some non-deterministic wiggle room in remote controlling the full graphical client), so you really want to cover all assertions together for each run.

Re:Automated Testing for League of Legends by finishingmove on Sunday May 01, 2016 @11:02PM (#11608987)
They should be tested independently, each with a completely reproducible setup that does only what is required for the assertion that will be made. I'm not sure what's hard to understand here... Are you trying to say that they need to be run together? Because it doesn't seem like that to me.
Re:Automated Testing for League of Legends by dimino on Sunday May 01, 2016 @05:48PM (#11608009)
The code itself isn't super duper pythonic, which is not ideal, but definitely the right direction for a project like this to move.

It's just hard sometimes to set and follow standards when you're pushing to demonstrate value. Not impossible, but hard.