I delete dotfiles so infrequently (once a year?) that this just isn't a problem for me.

I also don't have all dotfiles under version control, just the ones I've essentially "built myself", so maybe that makes it easier.