Do let me know if I misrepresented it. I thought it was a clever way to show that the models are not finding inherent relations in the data, by just randomly labeling all training data and having the same speed/accuracy on the random labels as on the real ones.
Edit: I worded the above poorly. They showed the models were capable of merely learning the training data. Despite that, they also showed that moving the trained models from test to validation did not have a linear relationship with the amount of noise they added to the data. That is, the training methods seem indistinguishable from memorization, but also seem to learn some generalisations.
Do let me know if I misrepresented it. I thought it was a clever way to show that the models are not finding inherent relations in the data, by just randomly labeling all training data and having the same speed/accuracy on the random labels as on the real ones.
Edit: I worded the above poorly. They showed the models were capable of merely learning the training data. Despite that, they also showed that moving the trained models from test to validation did not have a linear relationship with the amount of noise they added to the data. That is, the training methods seem indistinguishable from memorization, but also seem to learn some generalisations.