At least personally, it's the insight into human brain mechanics and new abilities to test hypotheses in this field that gets me most excited about deep learning developments, rather than the improvements in performance on various tasks.