There must be some granularity at which voxel engines can work smoothly today. Let's say my room is a square with 320 pixels on each side. 320^3 = 32,768,000 voxels, which looks like a lot, but an RTX 2080 can render 17 billion triangles per second, which works out to 273 million per frame at 60fps.
Let's ignore gravity and focus on momentum vectors and collisions. Even if simulating physics for each voxel were 10x slower than rendering a triangle, you could do 320^3 voxels in 60fps on today's hardware.
And this is naive, assuming you can't merge neighboring voxels together! I think you're being unfairly pessimistic.
Current systems simulate about 30k simple bodies. So that is already off by 3 orders of magnitude.
And that is a basic simulation, nothing complex, no fancy physics simulated, just rigid collisions.
Even if you could simulate the numbers you propose, 320 voxels per side is really low. Each voxel would be around a centimeter wide, which would look very bad (compared to the usual way of doing things).
There is no way we can simulate physics at the level you suggest for real-time games, not even in a hundred years. Unless you achieve a breakthrough in fundamental physics which would have an impact in every aspect of life, not just games.