You just have to include all the parameters of the animation as part of your "dirtiness" calculation. I reckon in practice this means centralising pretty much _all game state_ in some way that you force the dirtiness state to be set correctly.