Not necessarily.
"Excuse me. Is this your phone?"
Or some derivative of that.
You only need to look at the phone for a brief moment. It's designed to quickly unlock. If you had to stare at the phone for 10 seconds it would be a frustrating experience.
Should make it more difficult (though not impossible) to force an unlock by waving the phone in an unwilling person's face?