Decoding the “changes” can be dependent on the “changes” in a previous frame, notably that the prediction for motion vectors can come from previous frames’ motion vectors. Which I frames obviously don’t have.

Besides that the only way to losslessly re-encode frames is to use lossless encoding, which would easily be a 50-100x size bloat.