>One reason it's difficult to imagine other people have the visual imagination they think they do is because I so rarely see people just sitting and imagining things for long stretches - which I assume would be borderline irresistible if you had the kind of fantasy machine to show you whatever you wanted. Perhaps people do this in private and don't talk about it much?

I'm guessing we're wired by evolution to put less value on the imaginary since those prone to it wouldn't have done as well. Some people, however, are prone to spending their time in imagination and there's some research on it as a mental disorder. See https://en.wikipedia.org/wiki/Maladaptive_daydreaming.