There was similar work by W. Zaremba (OpenAI I think). LSTM trained to interpret python source code. Took a ton of hackery to get it to not fall apart on simple operations and it continued making indexing errors. I wouldn't read too much into these results.

https://arxiv.org/abs/1410.4615