The author does understand what's happening, but knowing things consciously doesn't stop other feelings from subconsious parts of the brain from having an effect. Though I think one of the author's biggest concerns comes from thinking about this from the perspective of someone who doesn't understand how the chat bot's model works. It would be easy for some of those people to feel like they actually do have a human connection with the AI.