Using the wrong one can activate a low-level fear response for people trying to "pass" on the binary, but it's easily quelled by generally being kind. The fact that you're concerned about being wrong tells me you aren't the kind of person they worry about. Conventions wouldn't hand out pronoun stickers and pins if the widespread expectation was that people would just know.
Most of the social pressure you perceive comes from well-meaning cisgender allies trying to use their privilege to mold the world into something a little safer. Sometimes they go overboard. Have you ever seen a cisgender person try to argue with a nonbinary or binary trans person about their own identity not realizing they're talking to a person they're arguing about? It's surreal. They mean well.
I was friends with a couple of trans people on college, and I would always either directly address them as "you", or use their name where I would normally say "he/she". It's not a big deal, I just hate that there is a subroutine in the back of my brain shouting "Don't say the wrong word! Don't say the wrong word!". I think that's the cognitive load OP was talking about.