Might help. You could also allow it to output edits instead of just a sequence. Probably have to train it on edits to make that work well, and the training data might be tricky to obtain.