You should never have 10% of your unit tests depend upon the location of a menu. You should have one unit test that depends upon the location of the menu, and everything else should be isolated by stubs, shunts, mocks, whatever. Otherwise, they aren't really unit tests, because they're testing more than a unit.

I didn't realize how wrong I was doing it until I got to Google. (Then again, a lot of Google's best programmers are doing it wrong too.) Perhaps this is because doing the right thing - mocking out your dependencies and testing only one feature per test - is often harder in the short term than just running all your code when you test the topmost layer. But I think the point needs to be hammered home: most programmers who're doing "unit" testing are not doing unit testing, and their code would be much less brittle if they took the time to stub out lower layers.

I think it depends on the project - is it a 'design up front, go away code once' project, or an iterative constantly changing thing.

1) You don't bother testing much. You make lots of changes, iterate quickly. Before the changes, you have code that, let's say, you know works. After the changes, you know there's probably bugs, but you don't know where.

2) You test fairly thoroughly. You make lots of changes, iterate quickly. Before the changes you have code that you know works. After the changes, you know where most of the bugs are because your tests tell you where they are.

It's the difference between walking with your eyes open or closed... it may be more work to process all that information about obstacles, but it sure is helpful if you want to avoid those obstacles.

1. Design it right the first time, tests and all.

2. Build a throwaway, no tests, little refactoring as you quickly iterate

Programmers who fall into category 1 in my experience mistakenly believe that they understand the requirements of the software they are writing. Category 2 programmers tend to be ones who actually deal with users/real world, and know that they don't understand the requirements.

My position is that if you are building something for the first time, you are going to get it wrong, so get it wrong as quickly and cheaply as possible. This means getting it in front of users as quickly as possible, bugs and all.

I often find that I unit test all but the latest layer. As soon as I start depending on code from something else, it's time to write some tests. But if I just wrote something, it's probably in flux and tests would just prematurely lock down the interface. Write the code, write the client (which probably requires that you rewrite the code), and then write the tests in lieu of debugging the code.

Hasnt it been done already? Couldnt find anything I would class as 'scientific' with a quick google search. Would be a good thesis for someone.

If I'm constantly changing my mind about how best to do something, I don't want the massive overhead of having to change unit tests all the time as well.

I'll test as I go, in the code.

Unit tests should hook into the application in such a way that you can reorder the GUI and change nothing.

If you are using Unit tests correctly then any change to the code base at worst requires 10% of that effort to change the test cases. If you can think of any meaningful change that is worse then that your something is wrong with how you're testing.

The tests tell you where the tests fail. Some of those failures may be due to bugs, and others may simply due to the way your code now works.

Which highlights one of the "problems" with unit tests: if something fails, you have to take time to go and fix it. If you have no tests, you can assume there are no bugs because you are such a great programmer, then go home after a job well done.

That there actually are bugs there doesn't bother you, because you don't know about them. It takes less time to write buggy code.

No. More often than not, if you change was a large architectural one, which is often the case in young software, your tests are now irrelevant, or broken.

>> "It's the difference between walking with your eyes open or closed... it may be more work to process all that information about obstacles, but it sure is helpful if you want to avoid those obstacles."

Do you seriously think people are just blindly coding and never testing anything here?

  while(true) {
    writeABitOfCode();
    while (!compile()) fixCompilationErrors();
    testThingsYouMightHaveChanged();
    checkLogsForIssues();
  }

If you change that into writeAnAutomatedTestForItAndRunYourTestSuite() you get the following benefits:

1) Every test you write accrues value onto your test suite and helps you avoid problems later;

2) You don't need to spend time to figure out what you might have broken with your change - your test suite tells you in seconds;

3) You can say with some measure of certainty that it's extremely unlikely that your change broke anything.

  while (!compile()) {
    fixCompilationErrors();
    fixUnitTests();
    rewriteUnitTestsThatNoLongerMakeSense();
    WriteNewUnitTests();
  }

Delete em. You should be iterating your unit tests.

Over time you will find some really useful unit tests for common mistakes, and others that are basically useless. Try and approach things from the perspective of what you would like to know works and not just tossing everything and the kitchen sink at the problem. When you find a bug add a unit test just in case it comes back, if you change major parts of the system, port the most useful unit tests and delete the rest.

I think of unit tests as saving you from the (I thought I just fixed this) problem.

PS: The major advantage of this approach is you keep adding more tests to the older parts of the system, which are also the most costly to change because other parts of the system assume it works. And assumptions you and your code is making can be really hard bugs to find and fix.

I think your line is too coarse. I do heavily rely on unit tests even in "iterative constant changing projects". But in the very early "sketching" stage of development I don't bother to write tests. I just play with code on REPL (I work with Common Lisp or Scheme most of the time), in a flat namespace. Then, once I start putting code snippets together into packages (CL) or modules (Scheme), that's about the time I start writing tests as well. After that, even I constantly change stuff, sometimes drastically, unit tests do help me a lot.

It is true that you need effort on reworking on the tests. Actually I think writing tests needs more work than writing actual code. And I think that's the right thing. Writing something that works is an easy part. Making sure it doesn't break in every possible situation is much harder. In language like CL, you can fairly quickly write something that takes typical input and reasonable output without serious thinking. To make sure it works in every corner case you have to think much harder and precisely, and test code reflects that thinking.

(NB: I know some programmers who seem to think out those corner cases in their brain and just spit out a beautiful code at the first attempt. They may not need support from test code, I guess.)

I put myself in the category of mediocre test writer, because I do bump into the problem fairly regularly of having a lot of tests fail when I do a rewrite. I'd love to learn how to write better tests...and your last paragraph seems to scratch the surface of a superior approach.

http://googletesting.blogspot.com/

http://diveintopython.org/unit_testing/index.html

These crazy rules people come up with for "proper object oriented" programming remind me of extreme religious rituals. You have to observe all these stupid little rules or else your code will become impure you and will spend an eternity in code maintenance hell.

I'll make a somewhat heretical claim even: Unit test are 20% useful engineering, and 80% fad. They are great for tricky code with well defined behavior (like a parser), but wasteful for most code.

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don't typically make a kind of mistake (like setting the wrong variables in a constructor), I don't test for it. I do tend to make sense of test errors, so I'm extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

http://stackoverflow.com/questions/153234/how-deep-are-your-...

Just wondering how long you've visited the region you're writing a travel review about.

Unit tests are about confidence in the software. Automation is the mechanism for sustained confidence. Engineering (and / or fad as you put it) is a vehicle to get there.

What's even more amusing is that the people prescribing these rules aren't even using languages where OO is implemented properly. I will start listening when they start using Common Lisp or Smalltalk for their examples. When they use C++ or Java, it just makes me giggle.

Also, are you objecting only to low-level unit tests, or to "integration tests" as well? (i.e. unit tests with a much larger "unit")

I don't TATFT. My metric for deciding what tests to write is "is it likely to break in a non-obvious manner?" If a breakage in this code will cause the application not to run at all, there's not all that much point in testing that. However, if the code is likely to result in some weird errors that will be hard to track down, I feel writing tests is a very good investment.

I work on a web browser, and every test is worth something. To some degree, we don't care what the test is testing. We want to be sure we don't change it unintentionally, thus breaking the web.

"They've just gone off the deep end, and I don't think these people write very much code if they're coming up with these principles, to be honest, it doesn't even make sense."

It probably wasn't meant that way, but it was taken as an unnecessary personal attack (despite Joel's frequent 'don't listen to me' implorations)

and why is this objectionable? Joel Spolsky thinks some agile "gurus" don't write much code.

Fwiw, I know at least a couple of these "guru" (note the quotes) folks who don't write production code and haven't done so in years and make their living telling other people how to write their code !

The correct way to counter this is to point at (open source) code they wrote (Bob Martin does this by pointing at the Fitnesse code base, so he got this part right imo), or if it is not possible, just calmly assert that Spolsky's thinking is wrong and leave it at that.

TDD is great if you use it appropriate along with good design - but "tests for everything" is just ideology. As he says, it's like the old OO schema, where you created these ridiculous inheritance trees and unnecessarily complex diagrams and such.

* selling software people pay money for

* writing software people install -- that is, /not/ web stuff

Then under what circumstances would it make sense to ship untested code? When is there a difference between code that has unit tests, versus that with only end-to-end tests?

And, BTW, I never found Joel Spolsky a particularly insightful person.

When I layer my tests I rarely end up with thrashing tests. Multiple tests often fail, but it's easy to figure out which test to focus on. And fixing the right test case fixes them all.

Even if it makes sense to write a test for menu placement, it rarely makes sense to write more than one.

Psht, why do that when you can just ad hominem for 3 paragraphs?