It's possible to give a person posthypnotic suggestion that number four does not exist, for example. When they try to count their fingers, hilarity ensues: 1,2,3,5,6,7,8,9,10,11. WTF! 11 figers, this is wrong. I try to count fingers in both of my hands separately and sum them together. 1,2,3,5,6 and 1,2,3,5,6. 6+6=12 WTF, now I have 12 fingers.

If math and logic were hallucinations, they would lead into contradictions.

You could just keep adding spatial dimensions until 12 fingers are predicted by the theory. Of course, everybody knows you have both 11 fingers and 12 fingers, depending on how the hand is observed.

The existence of contrary hallucinations does not imply that all hallucinations are contrary.

There's an old saying, if a problem seems insoluble, perhaps you are asking the question the wrong way. It ties (ahem) in with the parable of the Gordian knot. We've got a few seemingly insoluble questions (such as the nature of consciousness, the unification of gravity and quantum theory, etc) that are just screaming at us.