I mean, in principle I don't disagree. A quantitative kpi that shows before and after is usually measured differently for each product you're building docs for.

For example, if you have a developer tool, the number of devs you can successfully onboard after you have solid documentation up and running is an effective strategy for getting unimaginative middle managers to understand the value.

Generally the people I work with have already come to the right conclusion because they have tried everything else and not gained the traction they believe their product deserves.

Experimental docs withdrawal? That sounds a bit too extreme and harmful for any business. Unless you had a different technique in mind.

If their utility is in question, then the company does not consider withdrawing them extreme! If it turns out to be harmful, QED! When you perform a statistical test, you only have to expose as many users to it for as long as needed to achieve statistical significance. Not all users will be affected; you need to be able to make a comparison.

The foregoing assumes the alternatives are having docs or not. If the alternatives are something else, such as having different kinds of documentation, then the test treatments will be chosen accordingly.

Would you do the same with, say, the UI? Or an API?

Again, you have to define the alternatives. APIs have launch success criteria too. One way to assess zero-to-one initiatives, if that is what you are asking about, is to start measuring before the project is complete. For a UI, that could be as soon as it is usable but before all the features are implemented. Then you will be able to able to estimate the value of the remaining features.

You might like to read https://hubbardresearch.com/publications/how-to-measure-anyt...

The need for some things to require statistical evidence and for others to be accepted wisdom, devoid of evidence or maybe a hand wave at most, is a form of politics itself. It is why we have agile, open plan offices and shite docs!