Yes, vacuum energy. But that's still probability distributions, and so almost all of the vacuum is empty at any given time.
Most physicists would likely say that there's no point trying to visualize this stuff in mundane terms. Or at least, that's what they were saying the last time I took a physics course.
Granted, I don't really understand the details of what this means, but that's what I remember of the current science.