The dataset is a controlled variable in an experiment so it has to be held constant. If you update your model and the dataset for every trial (eg new hyperparameters or new architecture), and find it performs better, you won't know if the model is really better or just the dataset.