News for hackers. Stuff that matters.
Re:How should we evaluate progress in AI? by jononor on Monday July 16, 2018 @08:30AM (#17540741)
The cited example was attempted reproduced and found wanting. A plausible reason for this was discovered, and is accessible online. This in much better than no replication being performed at all. But yes, we need to continuously up the game. Adopting a standard that ensures model stability would be a good next target, and not accepting papers that don't uphold it.

I'd also like to see open-source runnable code the default for paper acceptance, with some sort of 'punishment' for not having it. Maybe even make it prerequisite of empirical papers.

There are no comments on this comment yet.