Not the person you asked, but I think it comes down to past experiences/family environment/etc. Theres poking fun at someone to signal "i like you anyway"... that is real. There's also people who cross the line with their words, and use "i was just joking" as an excuse to be cruel.

If someone has experienced a lot of the later, it makes sense that they don't really trust the former.