I don't think that's fair, any more than it would be fair to say mathematics is the hypothesis phase either. Mathematics might be used to build scientific theories, but mathematics itself is self-contained and doesn't need to reference the physical world at all. The goals of mathematics, the methods mathematicians use and the results they produce are largely orthogonal to the physical world.

This might be less true for areas of applied math, but that just shows how the boundaries of any field are fuzzy. The same for philosophy: most of it deals with questions—and, subsequently, arguments and answers—that are fundamentally distinct from science. There are certain areas classically part of philosophy that have become the subject of physics or psychology or the social sciences, but these parts do not characterize philosophy as a whole.

It's not controversial, it's just not what philosophy is I think.

Far be it from me to be able to give a precise (much less consise) definition of what it is exactly, but it's not merely the hypothetical phase when doing science.

Immediately metaphysics comes to mind as a philosophical field that is 'unscientific' in the sense that sometimes(mostly?) advances are not made by empirical measurements in an axiomatic system, but by convoluted and extreme thought-experiments with 0 experimental underpinnings.

Another example is pointed out above, in the philosophy of mind, about the fundamental difference between seeing an experience happen (for example with a brain scan/direct measurement of neurons) and actually experiencing something. That is not something that you can easily touch with science/surveys/brainscans/etc.