Yeah that’s how I imagine it would have to be done (scene capture from many angles + 3d scene recreation) but I disagree on “that’s never going to work”. The number of everyday things that people in say 1950s would think were “never going to work” is staggering.
My guess is that very few watchers would prefer a 3d reenactment over the 2d original stream, as the reenactment will always have less details or even show things incorrectly.