I think a better approach would be to get a dataset of songs along with their acappella versions and fine tune the network to that.