There have been quite a few cases of the neural nets spitting out the code they've been trained on verbatim, including comments IIRC. They're not just "learning" (if they're "learning" at all).