Well, whether it is or isn't depends on how good the docs are and how well you write them. Yes, in many cases it can be easy to forget if the documentation is not woven in well enough.
> All public APIs should have documentation, even if you believe it's obvious what they do.
As a note part of the function of such documentation is to establish standards for what is acceptable in terms of expected input and output handling. What this means is that if documentation defines the code contract, then the first thing you look at when debugging is the API's documentation. Then, if it matches what you are doing, you might dig deeper.
What this gives you is not debugging by comments (something K&R rightly hated) but asking which side the violation of code contract is on. If the documentation doesn't match what you are doing with it, then the violation is on your side. If it does, then the violation may be on the API's side. The goal here is to define where changes can most productively be made.
> Because there's other code in the wild that relies on the behavior that you promised.
That's exactly right. More specifically the API documentation is the promise.
> If you're changing it so often that updating the damn comment is an issue you either document implementation details that don't belong in API documentation or your API is unstable crap and nobody should be using it.
The thing is it took us a long time to get our documentation approach right in LedgerSMB. It was a struggle that really only I think reached something I am happy with 5 years into the project. A lot of our public SQL API's are not documented actually, because they are dynamically discovered at run-time and are minimalistic (and consequently the developer contracts far more vague than the API conventions, so it isn't always clear what belongs in the documentation since it is all dynamically looked up anyway), but our Perl code is very well documented and I am very happy with that.
For the SQL though, it's written with documentation generation scripts in mind and therefore the question is what you can document on top of what is already there in the system catalogs.
All I know is, I help maintain a very large set of public APIs that my team is very resistant to changing, and yet somehow the docs are still out of date.
But that can be a valid complaint as well. There is an undeniable cost of maintaining tests, and it shouldn't just be taken for granted that the cost is worth it.
The question is not whether to test or whether to document, but what to test and what to document.
See http://api.jquery.com/jQuery.ajax/ if you want an example of on API that could easily have a few typos in it. Who is checking to make sure it's 100% in sync with the code 100% of the time? Not trying to say this example has bugs in the docs but there's a LOT of behavior described that could be out of date.
Now sometimes one comes to the conclusion that an API is broken, so you have to modify the comments, and then modify the code, but there is a reason to do it in this order.
When you modify the comments you are modifying a set of promises you have made to other coders. This allows you to think through how this change is going to work and how it will affect other code out in the wild. Then, when you modify your code, it is going to be better.
Oh how do I wish this were the case!
> Documentation that can break visibly when things change is better than static documentation.
Agreed, but I would not call your average run-of-the-mill unit tests "documentation", nor can all documentation can be programatically tested.
> Oh how do I wish this were the case!
It's certainly not always the case, but it's a whole lot more likely to be the case than for unchecked documentation.
> > Documentation that can break visibly when things change is better than static documentation.
> Agreed, but I would not call your average run-of-the-mill unit tests "documentation",
I think "is it documentation" is probably more of a spectrum than any particular threshold, and run-of-the-mill unit tests probably do fall on this spectrum though I'd probably agree that they're not particularly far along it (though that surely varies with the habits of those writing the tests).
> nor can all documentation can be programatically tested.
As a practical matter, that's certainly currently the case - tooling is not set up for testing documentation, and there are things we'd want to check that would be hard to check in any event. Theoretically also, there are certainly properties that can't be statically demonstrated. I'm not entirely convinced that there's nothing we're interested in that couldn't eventually be got at for the programs we care about, though it's certainly a possibility. Regardless, it seems an ideal worth pushing towards, and if tested documentation is interwoven with untestable documentation such that some conceptual locality is preserved it's less likely (though absolutely still possible, to be sure) that you'll forget to update the other when you're forced to update the one.
It isn't.
All public APIs should have documentation, even if you believe it's obvious what they do. This documentation never goes out of date because once you release your API it tells you what you cannot change. If you changed the code so that your documentation is now wrong - this code change is a bug and you should fix it. Because there's other code in the wild that relies on the behavior that you promised.
Of course in practice you do have to change that behavior every once in a while. But this should be a big deal (that usually includes bumping up version numbers, mentioning it in release notes, etc). If you're changing it so often that updating the damn comment is an issue you either document implementation details that don't belong in API documentation or your API is unstable crap and nobody should be using it.
PS. Complaining that API documentation gets out of sync with the code is like complaining that unit tests break when you change the code. Duh - that's what they are there for!