Re:The testing pyramid should look more like a crab
by vosper on Tuesday November 03, 2020 @05:21PM
(#24984699)
> The Cypress e2e tests on our build server sometimes fail for no discernible reason.
You'll probably be interested to hear that Cypress are... uhm... addressing this by adding "flaky test detection" and some amount of automatic retrying in v5.
We use Cypress at work, we have flaky test problems, but I'm not convinced much of it is actually Cypress's fault. As others have mentioned, writing front-end code to be testable by E2E is not necessarily straightforward, and when you're trying to add tests to a big existing codebase I think you kinda have to accept some flakiness and retrying.
What I've been wondering about recently though is: if you construct applications in a certain way do you still need real E2E tests? For example, React is declarative, and with a state-management system like Redux shouldn't you be able to get the same kind of assurance that E2E gets you without needing an actual browser? Maybe things like animations, visibility etc are where that comes unstuck?
You'll probably be interested to hear that Cypress are... uhm... addressing this by adding "flaky test detection" and some amount of automatic retrying in v5.
We use Cypress at work, we have flaky test problems, but I'm not convinced much of it is actually Cypress's fault. As others have mentioned, writing front-end code to be testable by E2E is not necessarily straightforward, and when you're trying to add tests to a big existing codebase I think you kinda have to accept some flakiness and retrying.
What I've been wondering about recently though is: if you construct applications in a certain way do you still need real E2E tests? For example, React is declarative, and with a state-management system like Redux shouldn't you be able to get the same kind of assurance that E2E gets you without needing an actual browser? Maybe things like animations, visibility etc are where that comes unstuck?