the nature of the label bias present in rnn - the weights aren't learned by estimating the loss over the whole input/output, they are localized... representation helps a lot for the decisions but if you trained rnn-s differently - using the loss over the whole output - you would get better results