I think that's could be focusing too much on the "inner voice" and assigning that the "you" label. At least for me I have a normally chatty inner voice, and usually think of that as "me", but if I step back I realize there is a lot more to "me". There's other thoughts/knowledge/processes happening that can bubble up. At least my interpretation is the voice is just the aggregator, but there's really multiple systems coming together.

If I switch tasks and do something very creative, say playing music or drawing, the inner voice goes away and I can enter a "flow" or meditative state where things just happen. It's a very different mode where it's just feelings/emotions/imagery and no language based thoughts.

Also when I get really really tired, but power through it, I have a "recursive observer" where I kind of view myself on a delay and analyse everything I do, it's kind of annoying.