It's extremely common. You want to know that objects/rows from one test don't bleed into another by accident. It allows you to write more strict and simpler assetions - like "there's one active user in the database" after a few changes, rather than "this specific user is in the database and active and those other ones are not anymore".
Leaking information between test runs can actually make things pass by accident.
Re:A ChatGPT mistake cost us $10k
by zeroonetwothree on Sunday June 09, 2024 @09:02PM
(#40629035)
It makes your tests more robust. Generally you don’t want tests that are too sensitive to external state since they will fail spuriously and become useless.
Re:A ChatGPT mistake cost us $10k
by brigadier132 on Sunday June 09, 2024 @09:22PM
(#40629176)
Of course your tests shouldn't be sensitive to external state. Why would other tests running affect your test?
They shouldn't. But they do. We're not perfectly spherical developers and we all make mistakes. Sometimes it's also extremely tricky to figure out what state is leaking, especially is it's an access race issue and happens only for some tests and very rarely. If you haven't seen that happening, you just need to work on larger projects.
Re:A ChatGPT mistake cost us $10k
by brigadier132 on Monday June 10, 2024 @11:45AM
(#40634817)
I've worked at several large companies tyvm. In our e2e tests we did not isolate dbs. If the test failed because of other tests running that's a bug in the test and that person would get automatically @mentioned on slack and they would have to fix the build.
Why? In fact, not having good isolation would have caught this bug. Generate random emails for each test. Why would you test on a completely new db as if that is what will happen in the real world?