If you’re integrating/being an “expat” in a society, you’re going to develop familiarity. The “average” Korean won’t know anything about you and continue to just treat you like a tourist (unless you speak fluent Korean), but your social and professional circle won’t. You can’t go very far into relationships in a foreign country without people knowing your background.

Less importantly but somewhat obvious: many Koreans are fairly competent in English and familiar with common American accents. They’ll know pretty quickly if you aren’t speaking with one.