News for hackers. Stuff that matters.

A comment on Findings from the Imagenet and CIFAR10 competitions. Parent comment

Re:Findings from the Imagenet and CIFAR10 competitions by jph00 on Wednesday May 02, 2018 @05:17PM (#16981232)
No I haven't tried yet. One of our students has started experimenting with language models (AWD LSTM) and getting encouraging results (about 3x speed improvement), but not with attention.

I really hope that these results will help encourage more people to both see where and how super-convergence can be achieved. I suspect that we're only scratching the surface of what's possible.

[ Parent | on HN ]
Re:Findings from the Imagenet and CIFAR10 competitions by cs702 on Wednesday May 02, 2018 @05:36PM (#16981365)
Thank you for the prompt answer -- and for your intellectual generosity.

I too hope these results encourage more people to see where/how super convergence can be achieved.

FWIW, trying to achieve it with fully attentional models is on my "R&D things to try" list at work.

[ Parent | on HN ]