News for hackers. Stuff that matters.
Re:The day I started believing in unit tests by bedobi on Tuesday December 19, 2023 @10:03AM (#38696373)
unfortunately legitimate use cases for unit tests (like this) are pretty rare

in corporate codebases, overwhelmingly, unit tests are just mocked tests that enforce a certain implementation at the class or even individual method/function level and pretend that it works, making it impossible to refactor anything or even fix bugs without breaking tests

such tests are not just useless, they're positively harmful

https://gist.github.com/androidfred/501d276c7dc26a5db09e893b...

Re:The day I started believing in unit tests by nabbe23 on Tuesday December 19, 2023 @10:15AM (#38696585)
Well-written breaking tests represent something changing in a code base. You can be intentional about breaking a test, but then at least you can be very explicit about what you are changing.

Have seen all to many times I've broken a unit test in a code base that I did not intend to break, just to have an aha moment that I would have introduced a bug had that test not been present.

Unit tests are a trade off between development speed and stability (putting aside other factors, such as integration tests, etc). In large corporate settings, that stability could mean millions of dollars saved per bug.

That example you provided is a poor one and not really consistent with your point that unit tests are useless - the point is being made that that specific test of UserResource is useless, which I also agree with. Testing at the Resource level via integration test and Service level via unit test is probably sufficient.

Re:The day I started believing in unit tests by Joel_Mckay on Tuesday December 19, 2023 @10:20AM (#38696659)
Especially true if you get emergent side-effects from non-obvious shared state dependencies in large projects.

Nightmares... =)

Re:The day I started believing in unit tests by nabbe23 on Tuesday December 19, 2023 @11:18AM (#38697488)
Yes sir :)

And pragmatically - this always happens at some point. Something something about deadlines and need to get this out yesterday.

Re:The day I started believing in unit tests by Verdex on Tuesday December 19, 2023 @10:21AM (#38696676)
I've been on several projects where we had a significant number of unit tests that would only fail when requirements would change and we had to change the code.

"Look, if it only fails with requirement changes, then maybe we're better off not having them."

This only made people uncomfortable. They just don't like walking down the mental path that leads them to the conclusion that their high coverage unit tests are not worth the tradeoff. Or even that there is a tradeoff present at all.

Meanwhile, PRs constantly ask for more coverage.

Not very often, but sometimes someone will mention: "but unit tests are the specification of the code"

  test ShouldReturnOutput {
    _mock1.setup( /* complicated setup code returning mock2 */ );
    _mock2.setup( /* even more complicated setup code */ );
    
    let output = _obj.Method( 23.5 );
    
    Assert( output == 0.7543213 );
  }

  /* hundreds of lines above this test case */
  setup {
    if ( _boolean ) {
      _obj = new obj(_mock1);
    }
    else {
      _obj = new obj(new mock());
    }
  }
I'm just not sure I can get there.
Re:The day I started believing in unit tests by bobthepanda on Tuesday December 19, 2023 @10:14AM (#38696556)
The point of unit tests is not to CYA during refactors, but to confirm that the implementation is consistent between small changes without weird side effects.

A coworker once thought unit tests were dumb, and ended up writing code that repeated the call to an application 10x for the same info. This didn’t result in a changed UI because it was a read, but it’s not good to just suddenly 10x your reads for no good reason.

TFA also describes discovering weird side effect race conditions as a result of unit tests.

Re:The day I started believing in unit tests by bedobi on Tuesday December 19, 2023 @10:24AM (#38696704)
> confirm that the implementation is consistent between small changes without weird side effects

not sure what this is referring to, but I'll give an example

say you have a requirement that says if you call POST /user with a non-existing user, a user should be created and you should get a 2xx response with some basic details back

you could test this by actually hitting the endpoint with randomly generated user data known to not already exist, check that you get the expected 2xx response in the expected format, and then use the user id you got back to call the GET /user/userId endpoint and check that it's the same user that was just created

this is a great test! it enforces actual business logic while still allowing you to change literally everything about the implementation - you could change the codebase from Java Spring Boot to Python Flask if you wanted to, you could change the persistence tech from MySQL to MariaDB or Redis etc etc - the test would still pass when the endpoint behaves as expected and fail when it doesn't, and it's a single test that is cheap to write, maintain and run

OR

you could write dozens of the typical corporate style unit test i'm referring to, where you create instances of each individual layer class, mocks of every class it interacts with, mocked database calls etc etc which 1) literally enforce every single aspect of the implementation, so now you can't change anything without breaking the tests 2) pretend that things work when they actually don't (eg, it could be that the CreateUserDAO actually breaks because someone stuffs up a db call, but guess what, the CreateUserResource and CreateUserService unit tests will still pass, because they just pretend (through mocks) that CreateUserDao.createUser returns a created user

Re:The day I started believing in unit tests by Verdex on Tuesday December 19, 2023 @10:40AM (#38696955)
To be fair to unit tests, I really like them for making sure that complicated code gets tested thoroughly. However, very often the complicated code isn't isolated to a single unit. It instead lives distributed amongst multiple objects that are reused for several competing aspects and all have their own undocumented assumptions about how the world works.

Now maybe this implies that we need a wide scale change in coding methodology such that the complicated parts are all isolated to units. But pending that, I'm not sure that the answer is a bunch of static mocks pretending to be dynamic objects with yet another set of undocumented assumptions of how the world works.

The unit tests that have made me happiest has been unit tests on top of a very complicated library that had a very simple api.

And on the other hand, the tests that make me most believe that the projects I'm working on are correct have been integration tests incorporating a significant part of the application AND the QA team's test very thorough test plan.

Re:The day I started believing in unit tests by bobthepanda on Wednesday December 20, 2023 @12:15PM (#38710968)
Unit and integration testing are not strategies to be used exclusively.

Integration tests relying on interlocking behavior are, by their nature, complicated. Unit tests are there to test what can be tested simply, and are cheaper to write, so your test structure should be pyramid shaped, with hopefully fewer tests as complexity increases.

Re:The day I started believing in unit tests by bobthepanda on Tuesday December 19, 2023 @10:38AM (#38696920)
I literally gave examples in my example.

As a general example, a unit test is great for things like:

Your ExampleFactory calls your ExampleSerivce exactly once, not more, not less, so you can check that side effects don’t result in unnecessary extra calls in more load.

This is particularly relevant in a language like Java; modern Java style is functional, but the old style relied heavily on side effects, and they’re still possible to write unintentionally.

Re:The day I started believing in unit tests by bedobi on Tuesday December 19, 2023 @10:49AM (#38697073)
I understand your example and I agree that in that particular case, maybe a unit test to ensure a given service calls a dao once and only once or whatever is justified

but I don't think the hypothetical risk of someone needlessly calling the db ten times is a good reason to justify adding that style unit tests to everything by default - if it happens, sure, add one for that particular call and call it a day

Re:The day I started believing in unit tests by Verdex on Tuesday December 19, 2023 @10:48AM (#38697058)
I believe the kind of scenario that bedobi is referring to is something like this (using your example):

Unit test exists, ExampleFactory only calls ExampleService once.

Hmm, it turns out that ExampleUIButton calls ExamplePoxyNavigator more than one time if the text in ExampleUIButton happens to be wider than the default width for the button.

What does ExampleProxyNavigator do? Oh, it calls ExampleFactory which calls ExampleService. But only when the whole system is wired up for deployment.

The unit tests indicate that the system should be functioning okay, but when you put everything together you find out that the system does not function okay.

Re:The day I started believing in unit tests by domano on Tuesday December 19, 2023 @10:20AM (#38696664)
Using mockserver etc. you can cover for these things in component-test cases even more easily through your whole application while being more flexible with bigger code changes than unit tests allow.
Re:The day I started believing in unit tests by bobthepanda on Tuesday December 19, 2023 @10:35AM (#38696876)
A lot of places on the internet treat component testing and unit testing as synonyms. I’ve never heard of the former and it basically sounds like the unit tests as we write.
Re:The day I started believing in unit tests by Joel_Mckay on Tuesday December 19, 2023 @10:16AM (#38696596)
If maintained right, unit tests at edge conditions can quickly diagnose system and runtime state health.

If you work with malicious on incompetent staff at times (over 100 there is always at least 1)... it is the only way to enforce actual accountability after a dozen people touch the same files over years.

"The sculpture is already complete within the marble block, before I start my work. It is already there, I just have to chisel away the superfluous material." ( Michelangelo )

Admittedly, in-house automated testing for cosmetic things like GUI or 3D rendering pipelines is still nontrivial.

Best of luck =)