I used to always have this in the back of my mind, and would be hesitant to network with men because of this. I started asking people to meet up for breakfast, and would keep the topic of conversation about work/industry/etc. Once I got married, I noticed my paranoia with interpreting male co-workers intentions went down quite a bit.