> But if you just started a new project and you are still in a constant state of change then a large test suite will slow you down because you have to adapt all your tests all the time.

I have been in a new project where progress was slowed by all the tests. But the issue I have with "I will write tests later, after things have stabilized" is that, in my experience, that day never comes.

There is always a feature that would add value, or some fire to be put out.

Of course, you could arbitrarily pick a date or time to start testing, but as some one who was responsible for a smallish code base (30k lines, maybe 2-3 man years) with no tests, I can tell you that it is hsrd to start that. You have to either start testing when building a new piece of software (which is easy, but you really want to have assurances about the working of your existing code), or try to apply tests to previously untested code, in which case you are unraveling quite a bit of complexity and possible untestable/difficult to test constructs.

I write high level automated tests immediately at the beginning of a project, which have several upsides:

They can be read almost like project requirements, so they double as validation for solving the problems in the reqs.

They only use the outermost APIs of your code, which will change much less than internal module APIs, and so won't inhibit changes and experimentation as much.

They enable you to automatically test your code in the "real" scenarios that your project reqs describe, instead of just module-level tests that may not catch bugs in API usage of those "real" scenarios.

Later on in the project, as larger internal modules crystallize, you isolate those modules with their own outer APIs and write automated tests for their APIs.

Yes, this has downsides. High level white box testing exclusively during early stages obviously won't catch all your internal error cases early. However, I have found them to have superb ROI, especially in projects where time constraints have required pragmatic approaches.

YMMV.

Unfortunately, it's difficult to pinpoint the time to abandon that initial flexibility and create a test suite.

Moreover, creating that test suite after the project has grown is likely to be tedious, too... and often requiring serious refactoring to accommodate testing practices.

The XP folks were always all about changing requirements and flexible code, and unless they're all deluded, it's not true that testing necessarily hampers a new project.

When I start a new maven project, it comes with junit by default and with the test tree created. The first thing I do when starting a nontrivial class is to hit the shortcut for create new test. I should make it fail empty classes in the test tree. Automating the testing burden helps especially when interfacing with a tricky API; sometimes it's mocking, sometimes it's a real object in isolation.

I find tests to be invaluable especially in times of high churn and lots of changes. First they force me to think of my design in terms of components I can compose together into higher level systems, and when I find one of those components or systems is no longer how I want it I can make changes with my test suite telling me what I've broken, and what I haven't broken.

"But if you just started a new project and you are still in a constant state of change then a large test suite will slow you down because you have to adapt all your tests all the time. In the worst case you might even stop to improve your software just because updating your tests is too much work."

That's exactly when you need automated test suites, if your pushing multiple versions a day, then automated tests are really required.

Depending on how fundamental the changes are, I agree. However, usually when someone says that to me, its a huge smell that the unit tests are bloated and cover too much.

I'd argue the opposite - you should be very afraid of changing untested software because it is hard to keep it correct, and the more it changes the heavier the burden of manual testing (assuming you care to understand your code's behavior at all).

Then again, I skew towards integration rather than unit testing. So I am not locked into an internal design, but I know outward facing behavior is as I expect it to be.

> But if you just started a new project and you are still in a constant state of change ...

I don't understand what the rush is all the time. Sit down with a pad/pen or whiteboard and think about the design. the last mobile project i started, I sat down and wireframed on paper. Tests came naturally and didn't fight the design.

I think it depends on what type of software you are doing. If it is "just" a mobile project then you can pre-plan most of it. But I come from a more scientific background and most of the stuff I did back at university was pretty much impossible to plan because you did not actually know how you would solve your current problem. Usually you would try several approaches until you find the one that works. In such a situation tests are hard to use because they reduce your flexibility.

> ... most of the stuff I did back at university ...

I would place that kind of programming in a different class. Small, "one-shot" problems probably don't need a lot of planning.

That's the experimental / exploratory class of problems where you're not interested in the correctness of the process but results. AKA risk reduction.

For the other classes where there is "The Process" of validation, testing and approvals, a little planning up front saves some churn on the back end.

I have the same concern about iterative development. Seems like a lot of teams jump right into a coding sprint/iteration/cycle w/o a plan.

How do you verify that the lack of understanding of the form of the solution was due to the problem domain (scientific) and not your lack of exactness of confirmation and ability to see what is in the problem due to not actually knowing?

You are completely allowed to throw out your tests if you are doing this kind of exploratory programming.

Then, tests are simply your hypotheses.

Yes, that's one thing people often forget. It's OK to delete tests that are obsolete.

I work in an enterprise environment and with all the stakeholders involved it's impossible to get a decent design upfront. But I still think you can test a lot of things even while change happens. Just avoid getting too detailed.

Designing tests that need to be changed often is a sign of high coupling between your tests and your solution, which in turn is often a sign of brittle design. You are testing an implementation and not a ndesired feature. Another potential problem is that your interfaces are changing too much, which means you are having trouble with thinking it out.

Possible solutions to this are TDD, or trying to write your tests in such a way that they would fit any implementation of your interface. For interface design, outside-in TDD is an interesting tool.

In any way, your problems with tests at early stages probably come from lack of practice. Doing less tests is certainly not going to help.