I think ML people have crashed into the field of experimental analysis of algorithms. Rather than developing these kinds of heuristics, they’d be better served by reading what other fields have been doing for decades.

But I do like his last comment about trying to push author’s on the weaknesses. I might just blanket add to my reviews “can you describe more the possible draw backs” and see what they come up with…