Nobody reasonable is going to misinterpret a request for coffee as sexual harassment unless you are socially inept.

"Hey, do you want to get coffee sometime?"

"Oh, no thanks"

"Ok, cool. No problem"

Proceed to move on with your life.