Re:Signs you are doing continuous testing wrong
by macca321 on Wednesday November 07, 2018 @11:45AM
(#18400308)
'Test Pyramid' considered harmful IMO. It discourages people from writing tests against the public interface of their system , and instead encourages testing against internals.
This leads to a test pack that prevents refactoring, and can't be trusted to catch regression errors.
This is partly down to the widely confused definition of a 'unit test' (which a lot of people interpret as 'one test per class').
Re:Signs you are doing continuous testing wrong
by TarpitCarnivore on Wednesday November 07, 2018 @01:59PM
(#18401281)
TBH I feel like integration and UI tests can be combined at this point. Writing a bunch of tests to only check UI elements feels like a waste to me. I don't mean to go full on BDD where every step is spec'd out, but I've done it by interacting with elements and validating data with the API.
Re:Signs you are doing continuous testing wrong
by nobody271 on Wednesday November 07, 2018 @10:57AM
(#18399932)
I like test recorders (Selenium IDE). They aren't brittle if you manually enter your selectors. Plus you can hook a test recorder up to your own code to fill in any gaps. I do wish they showed mouse movement, though.
One I'd like to add is to keep it simple. When I hear the word pipeline I immediately think "oh no". I'm telling you man, some shops add a mind blowing amount of unneeded complexity with their pipelines.
Re:Signs you are doing continuous testing wrong
by alex__c on Thursday November 08, 2018 @10:39AM
(#18406994)
The objects under test are brittle and bound to change dramatically. Selenium can't offer you insurance against this.
I now remember a test suite we had on my project (automotive). We tested against the code structure... that is, we would follow a variable's lifetime and check that its value is correct throughout execution. Two releases later, guess which function got slightly refactored and threw off some 20 testcases?
This leads to a test pack that prevents refactoring, and can't be trusted to catch regression errors.
This is partly down to the widely confused definition of a 'unit test' (which a lot of people interpret as 'one test per class').
If you haven't seen it, please watch Ian Cooper's "TDD, Where Did It All Go Wrong" https://www.youtube.com/watch?v=EZ05e7EMOLM
Also, I thought continuous testing was having your test pack run all the time while coding, but hey ho.
One I'd like to add is to keep it simple. When I hear the word pipeline I immediately think "oh no". I'm telling you man, some shops add a mind blowing amount of unneeded complexity with their pipelines.
There's definitely some good with selenium, but I and others find it is often brittle and doesn't catch errors as reliably as would be ideal.
I now remember a test suite we had on my project (automotive). We tested against the code structure... that is, we would follow a variable's lifetime and check that its value is correct throughout execution. Two releases later, guess which function got slightly refactored and threw off some 20 testcases?