I'm also playing with RNNs for text generation - currently trying to make a Haiku writer that'll actually "make sense". In case anyone else is interested: https://speakerdeck.com/herval/creating-creative-machines-v0 (shared the code of the experiments too)