With bite-size integration tests, I find it's generally not too hard to isolate the cause of a failing test, because the code it's testing tends to be straightforward, and fairly easy to step through, if necessary.

I frequently have a harder time with unit tests. The code ends up involving a lot of extraneous abstractions that I need to think through. The test code is often so heavily mocked that it's hard to distinguish the behavior under test from stuff that's just being mocked or stubbed to get the SUT to run cleanly, meaning I've got to start with trying to figure out whether the bug is in the test or the code being tested.

It gets worse in long-lived code bases, where the unit tests are often subject to significant bit rot on account of how brittle they are. I've definitely had some code archaeology excursions reveal that the reason an entire suite of tests were tautological is because someone was doing a nominally unrelated refactor, and just put in the minimum effort necessary to get the tests to go green again.

You can argue that developers need to be more diligent. Me, though, I figure it's sort of like those lines of bare dirt you see criscrossing the lawns of university campuses: when things get to that point, it's a sign that the official way of getting around isn't appropriate to most people's real needs.

I do think it's important to have unit tests when the unit's behavior is complicated. Where I start to get worried is when there are unit tests being written against classes that have very little behavior that doesn't involve interaction with some other module.

I think this is precisely where unit test suits start to have problems. Good, flexible unit testing requires a lot of judgement about what will be useful to test and what will be too much of a burden in the future. Unfortunately judgement is hard to acquire and even more difficult to teach, and a lot of teams want to create and enforce over-dogmatic testing "standards." When unit testing, you have to balance:

1. What testing do I need to have confidence my code is working?

2. What testing do I need to catch likely regressions?

3. What kinds of tests will just get in my way in the future or are literally useless [1]?

[1] E.g. unit tests that essentially only test core language functionality, once you take out all the mocks.

So for a little passthrough/orchestration class, it probably doesn't make sense to do much testing. For something that actually performs business logic, that's a prime candidate for testing. I've seen plenty of tests that just seem to aim to increase coverage, heck, I've written plenty of those myself - but at the end of the day, the benefit they serve after being written is probably minimal.

You're only unit testing the code how you 'intended' for it to work at that time. Even though the tests are written, it probably wouldn't be uncommon for a bug to slip through when running your code, what you then can do is write another test to account for that scenario, then repeat and your code becomes more robust as a result.

Correct that the goal is to prevent regressions. I claim (I don't know how to study this) 80% of your tests will never fail and so they could safely be deleted - but I have no insight into which tests will fail so I say keep them all.

Incorrect because in fact it isn't hard to localize failures: it is something in the code you just touched!

Yes, but you also need to localize the effect of the bug to know why is it that the code you changed broke the program (and remember that we're talking about a case where the rest of the program is not familiar to you enough). Good unit tests can help you find the immediate effect of the failure, rather than the ultimate one.

Keeping below 1 bug per 100 lines of code is viable simply by being careful and thinking things through. That's a long way from perfection, but it really helps.

Once you can generally write mostly correct code then you can work on improving your process. But, until most of your functions are working when you right them just work on improving that.

Edit: And yea quality may drop as part of this transition, but you need to get a feel for how much you can pay attention to.

The tests are there specifically to help find the errors that “trying harder” didn’t catch. You don’t get a higher quality result by cutting QA.

You can build a factory that has a Great QA process that finds all faults and corrects them. But, inside the factory you still want to minimize the amount of defects for QA to find. Either way you still need QA, but it's easy to get into the habit of improving QA vs improving the initial process.

PS: One exercise I did for a while was every time I found a non syntax bug in a function I just wrote (aka bug at run-time) I would start over and rewrite it. It's painful, but 'Build' wait let's look at this again if I hit run and it does not work that's going to be painful. So is this actually correct? Anyway, doing that showed me how important it was to just focus on the code (and what it means) to exclusion of everything else.

But you will not drive significantly higher quality into earlier phases by telling your factory workers to try to do a better job. You drive higher quality by analyzing where your processes are failing you and changing them so that they no longer do.

If your workers are consistently failing to tighten bolts sufficiently, you can add a QA step to check the bolts every time, or require your workers use a torque wrench, or use a robot to make this more consistent. But you have to do something other than telling them to try harder. You undoubtedly told them to try harder already and it didn’t work. Now you need better tools or processes.

However, the lady designing the factory in the first place should know if tightening bolts or using an arc wielder has more issues. So sure, QA should include checking the oil level inside a robot in the factory to avoid problems in the first place. But QA is not all inclusive it's not for example part of the collage education that's training your workforce other than perhaps a requirement that they have said degree.

QA is the system of trying to produce at the desired quality. Anything done to improve quality is QA, including whether you hire better employees or train the ones you have. You can define QA more narrowly if you like, but then we're arguing semantics and we're already off in the weeds.

Heh...irony alert. You're advocating focusing on just getting it right the first time and you didn't get write right.

But seriously, the people you're responding to are correct. Any process that relies on humans being less error proneis bound to fail. You need to either create a process that makes humans less error prone (e.g. checklists) or embrace our propensity for making errors and plan for that eventuality.

> I find it nearly impossible to write a "unit" of code without also writing one or two bugs along the way

Finding bugs early is great, but minimizing bug creation is even better.

  1 3 2 5 9 2 8 7

  1 3 2 5

  9 2 8 7

PS: Now, their are a lot of tricks on how you can get better at this stuff. But if the parent poster tends to write bugs in most functions then simplifying things may help.

Sure, it's less efficient to write unit tests ahead of time for every case, but in a lot of cases fixing a bug faster when it's discovered is much more important than even double the hours spent during less "pressing" times.

I tend to agree - in my personal experience in organizations with strict TDD culture, a perverse incentive often emerges to preserve existing flawed architecture over obviously better solutions just because it's so painful to deal with all the tests.

One of software development's most powerful properties is the ability to iterate quickly: it's foolish to prioritize dogmatic beliefs about testing over that quality.

It's just an extension of your code, if you are going to throw some code away to change an interface, then throw the tests away too. If you are afraid to do that because of the time spent, then you probably spent too much time on writing tests.

While I am dogmatic about tests, I also believe that around 50% code coverage is normally enough in most application codebases. Cover the important parts, the parts that are hard to test manually, the "core" pieces that lots of other areas rely on, and some tests for bugs that you want to prevent happening again.

If you want to quickly iterate, go for it! You shouldn't have all that many tests in the parts you are changing frequently. But to change a core aspect of the codebase, or a really complicated aspect of it, then the extra work of rewriting the tests shouldn't be all that bad.

I'd shorten your last statement. It's foolish to prioritize dogmatic beliefs. Everything has a time and a place, and moderation is key.

This metric means very little. It does not measure the extent of code path coverage and much more.

Code coverage can tell you what code is not tested at all.

Now, this is very useful. But:

Code coverage can't tell you what code _is tested_.

Code coverage can't tell you how _well_ code is tested.