Re:StateOfTheArt.ai
by hooloovoo_zoo on Thursday December 06, 2018 @02:11PM
(#18621058)
For some reason they distinguish between accuracy and error rate for classification, and there is missing data for both. If you sort by error rate there are newer results for classification.
Yes. the idea is the community can help maintain it by adding results (and tasks or datasets)
I think this kind of resource is in a chicken-and-egg situation: for people to bother to contribute to it, it needs to be seen by others as an authoritative reference, and vice versa.
I agree that it's badly needed. There were other projects like this, that seem abandoned: http://rodrigob.github.io/are_we_there_yet/build/classificat...
This probably belongs on Wikipedia.
For example, the best result on CIFAR-10 for density models that I know of is 2.85 bits per dim (PixelSNAIL) rather than 3.35 (Glow), and this is a year old. PixelCNN++ is well known, and got 2.92.
For classification on CIFAR-10, current SOTA is much better than the 2015 Fractional-MP result (which also used non-standard augmentation, so perhaps not directly comparable to the rest)