Patch modules can be signed so assuming you only need signed patches (which seems reasonable) this shouldn't be too much of a concern.