Something you have and something you know, yes. But isn't the thing you have supposed to be separate from the client you're using to authenticate?

We're talking about the case where the phone is stolen. There's almost no reason to steal an authentication device other than harassment or tactical delay of access.