Additionally, most voxels would appear different with different view perspectives. Due to more or less voxels covering them.
The problem is that in 4D all voxels are visible to the viewer. So viewing a 4D apple would allow you to see the apple from all possible view points simultaneously, including interior views.
To me it just doesn't seem possible to replicate this concept in 3D VR.
Here's a video of a rotating 4d hypercube in a 3d perspective:
https://www.youtube.com/watch?v=t-WyreE9ZkI
It's called a tesseract, and as each face of a 3d cube is a 2d cube (a square), each face of a 4d cube is a normal 3d cube, that we see skewed by the perspective.
Your linked video maps the 4th dimension to time, it doesn't project to 3D. Projecting through time (especially non-interactively) lacks the immediate feedback needed for the brain to grasp it as intuition.
No, it projects it to 3d. The movements you see are rotations, not movements of an intersection plane. You can clearly see at any point in time each of the 8 identical cubes making up the tesseract, skewed and resized by projection to a 3d perspective.
I can represent 3D quite comfortably on 2D monitors, can there be an intuitive mapping of 4D to a 3D VR view?
I know 3D mapped to 2D suffers from occlusions and heavily relies on clues like perspective, shadow etc. But given enough time even a less intuitive 4D view could become intuitive with time, too.
edit: found this: https://youtu.be/S-yRYmdsnGs?t=252
even better: https://youtu.be/dy_MUfBuq2I (turn on subtitles)