I have Aphantasia, thankfully I'm invulnerable to this.

If someone attempts to degrade someone's emotional or mental state, that's something that should be stigmatized. But, that doesn't mean that the individual is not completely responsible for his own mental state, in the sense meant by Stoics. It's not about things, but our thoughts about things, which are ultimately under our control. We may play the victim, but we'd only be fooling ourselves.