We have no idea how this result would work on a new data point that has not been used in training.

It's bad statistics, bad data science.

> One round of cross-validation involves partitioning a sample of data into complementary subsets, performing the analysis on one subset (called the training set), and validating the analysis on the other subset (called the validation set or testing set). To reduce variability, multiple rounds of cross-validation are performed using different partitions, and the validation results are combined (e.g. averaged) over the rounds to estimate a final predictive model.

https://en.wikipedia.org/wiki/Cross-validation_(statistics)

You might object that it's difficult to achieve the level of inter-fold isolation required to make the technique sound, and indeed if you search the comments there's some question as to whether or not there was an information leak in their feature selection process. In that sense calling it "bad statistics, bad data science" might be reasonable, but it's also a powerful technique, so I don't think it's reasonable to dismiss out of hand without being more specific.