I have always felt the same. For me it is easy enough to have a little password book with everything written down. People often say what happens if there is a fire. I don’t have an answer to that but I have made it over 4 decades and not once had a fire so I’m going to believe the odds are I will not ever have a fire and chance it. Most things are recoverable with my phone number these days anyways. A much greater risk would be someone gaining access to my phone