Immediate thought - the network didn't learn sentiment in the training phase, it just clustered the data. The last phase was so quick to train because it was labeling already-clustered data, so a few data points in each cluster was enough to make it 'obvious'.
Actually it was only an output neuron, and for me it is the same if you call it cluster or not. It just learnt sentiment without any hint. Now I wonder if they can train it again and get the same sentiment neuron, that would mean something. Like sentiment is necessary to predict the next character? (word?)
For me this open my mind to new opportunities when training deep learning. For example I can do the same for images: train a network to recognize objects and later use the same network to predict sentiment or prettiness for example. And the best thing is that I don't need a lot of labeled examples for the second phase of training!