I totally get that. Personally though, I see conversational UI as an additional parallel mode of interaction, that you can use simultaneously. I can imagine e.g. coding, and interacting and thinking analytically about that, while still verbally being able to have a conversation with my computer for e.g. responding to email or calendar appointments. Not for everybody, of course.
Edit: I suppose, what I really mean, is that I'd like to interact with human things (email, meetings, messages, etc.) in a human way (conversationally, as I would a person), and I'd like to interact with machine things (coding, simulations, etc.) in a machine way. Trying to interact with human things in a mechanical way seems unintuitive, really, except it's all current technology affords, so we all have to learn these machine interfaces instead. And maybe that's why so many (non-techy) people find computers needlessly complicated.
As I understand it, we use different regions of our brain for interacting with tools, people, animals, etc. I enjoy using the tool part of my brain when computing. It feels "quieter" and contemplative. It feels like I am doing something, not like I am asking an assistant to do the work on my behalf.
I like social interaction a lot too, but as an introvert, it's kind of draining. I would have to leave computing if using a computer felt like being in an intense conversation with a computer-person all day. Even if the conversation is pleasant and fulfilling, it would be too much social interaction.