You misunderstand. The point is that "philosophy improves critical reasoning" is an empirical statement and hence requires empirical evidence. That doesn't mean philosophical discussions themselves must be empirical in nature. But when they refer to evidence, they do.

You're going to run into problems attempting to operationalize a variable like 'critical reasoning', in order to demonstrate, scientifically, its empirical validity. In fact, you'll need the help of philosophers, who have been studying various ways of interpreting what critical reasoning actually means[1], and how to properly operationalize variables[2].

Reasoning - using logic to form conclusions, inferences, judgements - is a process defined, taught, and professionally applied by philosophers. So, teach students philosophy and some other 'control subject'. They learn logic, rhetoric, and how to follow complex chains of complex thought.

It seems to be a difficult claim to make, that this experiment would some somehow not show an improvement in critical reasoning - however you operationalize it.

[1] https://plato.stanford.edu/entries/self-knowledge-externalis... [2]https://plato.stanford.edu/entries/operationalism/#OpeUniAna...

Alchemists are trained in how matter behaves and how to manipulate it... turns out they're not any more competent at it than anyone else. It seems obvious that a student of philosophy who has been trained in logic and critical thinking will be better at it than others, but obviousness is not in itself evidence. Lots of things sound obvious but later turn out not to be true.

The other thing is that you've made a dichotomy out of a quantitative problem: how much better are philosophers at it than anyone else? A little? A lot? Is it a skill that persists after graduation? Better at certain types of problems, maybe worse at others?

In any other context, on any other topic, a philosopher would never dare to make hand-wavey statements about how obvious and straightforward it is that X leads to Y. Only when it comes to their own field of study and its value do philosophers become sloppy and claim the value of their field is self-evident and frankly beyond discussion, or they try to turn it into an irrelevant meta-discussion about the nature of evidence. That's the irony I was pointing out. If you boast about being rigorous, be rigorous even when it doesn't suit your purposes.

The value of the philosophical disciplines of logic and epistemology to empiricism and science is as self evident as the value of algebra and numbers to mathematics in that you can't have one without the other. Empiricism and science depend upon logic, reason, and a particular method of obtaining knowledge.

You're also conflating rigor with empiricism. It is possible to be rigorous and be a rationalist, a phenomenologist, a theologin, or an empiricist.

Exactly right! I was about to say something similar but you phrased it much better than I would have :)