it's called two factor for a reason! You're suggesting a return to one factor, but ditching the PW and using the backup means of auth. What's supposed to happen is that you combine something you know (pw) with something you have (phone) as it's generally difficult for an attacker to get both.