Lemoine admits the model will say it is not conscience if prompted to. It'll also say it is conscience if prompted to. Lemoine says this is because it is a "people pleaser"[0], but to me that's begging the question (i.e. assuming the conclusion as a premise). It also seems plausible that the "AI" is just like a run-of-the-mill string-completing chatbot instead of being a people pleaser. Both are alternative explanations of the same phenomena, but one assumes it's conscious, and by doing so isn't making a compelling argument.

[0] https://twitter.com/cajundiscordian/status/15357826682476462...