A statistical analysis is the definitive answer to the skeptic's question "Prove that docs deserve more respect." And to be able to answer that question you have to observe what happens to business outcomes with and without docs then perform a statistical test to see what the difference is, judged against its cost. It's thankless work, I imagine.

Lobbying like you've done with this essay is a probably a better investment of your time. Now you simply have to make sure the right people read it and decide to do it :)

I mean, in principle I don't disagree. A quantitative kpi that shows before and after is usually measured differently for each product you're building docs for.

For example, if you have a developer tool, the number of devs you can successfully onboard after you have solid documentation up and running is an effective strategy for getting unimaginative middle managers to understand the value.

Generally the people I work with have already come to the right conclusion because they have tried everything else and not gained the traction they believe their product deserves.

Experimental docs withdrawal? That sounds a bit too extreme and harmful for any business. Unless you had a different technique in mind.

If their utility is in question, then the company does not consider withdrawing them extreme! If it turns out to be harmful, QED! When you perform a statistical test, you only have to expose as many users to it for as long as needed to achieve statistical significance. Not all users will be affected; you need to be able to make a comparison.

The foregoing assumes the alternatives are having docs or not. If the alternatives are something else, such as having different kinds of documentation, then the test treatments will be chosen accordingly.

Would you do the same with, say, the UI? Or an API?

Again, you have to define the alternatives. APIs have launch success criteria too. One way to assess zero-to-one initiatives, if that is what you are asking about, is to start measuring before the project is complete. For a UI, that could be as soon as it is usable but before all the features are implemented. Then you will be able to able to estimate the value of the remaining features.

You might like to read https://hubbardresearch.com/publications/how-to-measure-anyt...

The need for some things to require statistical evidence and for others to be accepted wisdom, devoid of evidence or maybe a hand wave at most, is a form of politics itself. It is why we have agile, open plan offices and shite docs!