They're trained to predict the next word, so yes. Now, imagine what is the most common follow-up to "Bye!".