Since you asked specifically about my residence to the first stop, which is obviously the portion of driving I perform more than anything else, I would say there's something almost... "Pavlovian" about it, in that the route and actions to get from my house to the first intersection are "burned in" to my brain, but not at all in a way I'd describe as being able to "visualize" it versus "describe it verbally," which is what I have to do in my mind if asked about it.
Whether it's significant or not (related to aphantasia), I have a horrible sense of direction, get lost while driving incredibly easily, and am entirely reliant on step-by-step directions to get somewhere (now that GPS in the car is a thing it's not too terrifying to try to drive somewhere I've never driven before, but I've been driving since before GPS was a thing available in cars and getting to new places was not fun back then!)
It's less instructions and more a full understanding of the structure at play.
> Also, it is near impossible to report objectively about your inner processes in a useful way.
I agree that discussing our mental/inner processes is difficult due to a lack of a common frame of reference on which to base descriptions. (I'm reminded of McCoy and Spock's conversation in Star Trek IV... "Are you telling me I'd have to die to discuss your experience of death with you?")
I (approx. 40 years old) became aware of the concept of aphantasia about three years ago. I immediately identified with it... or rather, I immediately found it mind-boggling that other people claim to experience anything close to "seeing" something by closing their eyes and thinking about it. That's such a foreign concept to me I had always assumed people's descriptions of visualizing things in their mind, daydreaming, etc were extremely metaphorical, not describing a process that is anything like "seeing" something.
I have talked to many friends at length about how they describe their "mind's eye" after realizing I was probably different. And even how it extends, in the case of some of my friends with particularly visual mind's eyes, to the "mind's ears" and "mind's nose" in music and scent recall.
Based on my friends' descriptions to me of how they try to describe how they experience mental visualization, it's clear the capability is a spectrum... some describe it as blurry gray image of what they're thinking about, some describe it as being a picture as real as a photograph that they can literally see and smell.
I agree with you that it's hard to tell how reliable the descriptions are relative to each other, but I can tell you this: hearing all of themselves describe it, even down at the "it's just a blurry gray 'image' in my mind" end of the spectrum, none of them sounded familiar to me or like any kind of "visualization" I've ever experienced while awake (specifically: none at all). If it were simply a matter of being able to accurately report objectively on our inner processes in a useful way, I feel like there wouldn't be such a clear divide between the way a minority of people (allegedly with aphantasia) describe it and the way the majority of people describe it. Whether it's aphantasia or not, it's clear to me at this point that there is definitely a "group A" and "group B" in terms of how people describe their "mind's eye" when asked.
Related, I also have a friend who says she has no internal monologue, another phenomenon I've read about. She thinks entirely visually. Not having an inner monologue is such an incomprehensible concept to me; literally the only way I can think is with my inner monologue.
Brains are weird.