Would you consider the voice an aural hallucination?
The inner voice, also referred to as the internal monologue, is a form of thought. It's the same as how daydreaming isn't hallucinating images, it's seen with the mind's eyes rather than the physical ones.
My inner voice is usually pretty reliable in giving feedback like: "dude, that made no sense, you clearly don't understand that" or "good job! You should write a blog post!"