Why not just restrict it to user-initiated events that can exit or enter full screen? Similar to how you have to press a button to enter full screen mode. It seems to make sense to do the same thing to exit, or other various actions.