News for hackers. Stuff that matters.
Re:10 years ago, someone wrote a test for servo that include... by dathinab on Monday April 20, 2026 @12:23PM (#47836536)
if it makes thing flaky

then it actually is a huge success

because it found a bug you overlooked in both impl. and tests

at least iff we speak about unit tests

Re:10 years ago, someone wrote a test for servo that include... by yxhuvud on Monday April 20, 2026 @03:40PM (#47839501)
This will often break on stuff like daylight saving changes, while almost as often you don't give a rats ass about the boundary behaviour.
Re:10 years ago, someone wrote a test for servo that include... by jstanley on Monday April 20, 2026 @02:20PM (#47838450)
Only if it becomes obvious why it is flaky. If it's just sometimes broken but really hard to reproduce then it just gets piled on to the background level of flakiness and never gets fixed.
Re:10 years ago, someone wrote a test for servo that include... by nomel on Monday April 20, 2026 @03:22PM (#47839260)
To get around this, I have it log the relevant inputs, so it can be reproduced.

The whole concept of allowing a flaky unit test to exist is wild and dangerous to me. It makes a culture of ignoring real failures in what, should be, deterministic code.

Re:10 years ago, someone wrote a test for servo that include... by marcosdumay on Monday April 20, 2026 @05:01PM (#47840637)
Well, if people can't reproduce the failures, people won't fix them.

So, yes, logging the inputs is extremely important. So is minimizing any IO dependency in your tests.

But then that runs against another important rule, that integration tests should test the entire system, IO included. So, your error handling must always log very clearly the cause of any IO error it finds.

Re:10 years ago, someone wrote a test for servo that include... by tremon on Monday April 20, 2026 @02:44PM (#47838771)
Burma-shave
Re:10 years ago, someone wrote a test for servo that include... by jamesfinlayson on Tuesday April 21, 2026 @12:30AM (#47844534)
I remember having a flaky test with random number generation a few years ago - it failed very rarely (like once every few weeks) and when I finally got to fixing it, it was an actual issue (an off by one error).