I find there's a balance to be struck. You want to get stuff in front of users quickly, but if that stuff sucks so much that it's unusable, you aren't going to release it (and if you did, nobody would use it). So you want enough tests that you can be reasonably confident of the code and not spend a lot of time debugging, since debugging is the real timesink that'll kill you.

I often find that I unit test all but the latest layer. As soon as I start depending on code from something else, it's time to write some tests. But if I just wrote something, it's probably in flux and tests would just prematurely lock down the interface. Write the code, write the client (which probably requires that you rewrite the code), and then write the tests in lieu of debugging the code.