I think you’re kinda right ... but maybe a better way to say it is something like:

Pitch can’t be termed an illusion, because it is a perception, not a fact of physics. So it is either always an illusion or never an illusion.

Frequency is a fact in the realm of physics. Pitch is something our mind labels things.