Well the whole "deep learning" revolution really only started around 2012 when the ImageNet competition was won with a neural net. There have been numerous small breakthroughs that collectively have made deep neural nets now easy to train, and neural nets now provide state-of-the art (and often human-level) results in areas such as image recognition, speech recognition and language translation.

There's also been a revival of reinforcement learning, especially when used together with neural nets ("deep reinforcement learning"), and again there have been many small advances that collectively make this work very well. This is the technology that powered Google's AlphaGo to beat the world champion at the board game of "Go", not to mention learning how to play many arcade games at beyond human level based only on the raw pixels and current score as input.

There have also been tremendous strides in AI hype leading folk to fear the robot uprising based on these more mundane machine-learning/neural-net breakthroughs!

Better training techniques. Some noteworthy advancements:

1.Dropout and its variations. Widely used in both vision and NLP

2.BatchNormalization and its variations.

3.Inception Style Cell.

4.Residual/Skip connections.

5.Better optimizers RMSProp/Adam.

The bigger news is actually the paradigm shift. Representation learning with gradient descent swarms the whole ML field, and becomes the new norm. End-to-end learning is vastly accepted and preferred.

As to GAN, it is very exciting in research, and has the potential to make itself a bigger deal than the previous listed advancements combined, under the condition we can make it works on sequence as well as on images, for now, it doesn't make a practical impact in applications.

Generative Adversarial Networks have managed to take the problem of sampling from a perceptual category and turn it into yet another stochastic optimization problem with end-to-end training of a black-box neural network.

So that's an advancement... I guess.

large public labelled datasets is probably the most overlooked one. the largest gains are in the areas with the most open labelled data.