That's quite close to saying that applied maths is not maths, which many people would disagree with. If you have definitions, theorems, proofs, I'd say that's maths, and you definitely have that in a sufficiently rigorous statistics course.

Of course, then there's the fact that statistics builds upon probability theory, and probability theory is, in a sense, a subfield of measure theory which in turn is about as mathematical as it gets (in the discrete setting, it also includes a fair amount of combinatorics).