Most sites solve this problem with a combination of 2fA and at least one security question.

Having my phone number isn't even good 2fA.