You’re right; it’s very inconvenient. Especially frustrating because I’m in the same boat as OP: I can absolutely hallucinate, but only if I’m on the edge of dreaming, or actually asleep.

I really wish I could change that. The machinery is clearly there.

Nikola Tesla was that way

I’m this way as well. Though I wonder to what degree this type of mental imagery ranges from person to person. To use a simple example, like placing something somewhere in my house and imagining how it would look. It’s not like I can actually see it at that location when I look there. It’s more like I am superimposing my imagination on the visual input from my eyes to create a composite representation of what it might look like. So it’s two systems working together, not like a full real image, if that makes sense. I see it in my “mind’s eye”. Curious to know what other people experience in this situation.

Yes, it's all in my mind. I don't actively hallucinate. But I can put the image in my mind I to the real world. I still don't see it but I can "put it there" and trace the outline, add or remove elements and see how the imaginary object would interact with the real world.

For example, if I'm trying to see if a piece of furniture would fit, I can use this technique. I might still need a reference like a tape measure since I can't just see on a spec sheet that the bookcase is 2 meters tall and 1 meter wide and know it will fit but I do have a pretty good conception of physical space. I often set my table saw fence by eye and when checking it with a tape measure it's within 2-3mm.