Here are some things we now do at Justin.tv because of experiences like the one you mentioned:
- Everything gets code reviewed
- We have started to introduce tests (sorry Bill, they're really quite helpful in the right circumstance)
- We have extensive monitoring for all systems in case something DOES slip through
- We have fully automated deployment (and rollback)
There's a cost to all of these things, either in setup time or in constant maintenance. When you have few users, maybe they're not worth it - as you grow they become essential.
I don't think I've ever claimed they're never useful ;) But I do still believe that, out of the things you listed, they have the lowest utility for the amount of effort involved.
Personally, I would do fully-automated deploy and rollback first, closely followed by monitoring, then code reviews, and then if problems were still slipping through the net I'd tell people to start writing unit tests.
I'm trying to bootstrap a startup now, and if it fails to gain traction then justin.tv would be one of the first places I'd interview. (Hey, I like watching competitive TF2, and they provide a hell of a platform for casts, can I say.) Now I know this blog post was sort of written for the purposes of recruitment, but it's sort of making me think twice about whether I'd want to interview there. Bleh.