SSH is flawed in that you have to trust the host the first time you connect to it. For most people this is a reasonable tradeoff, but for security nerds it can be an impossible barrier to adoption. This is why some SSH servers spit out big long hexcodes or weird picture things, you are supposed to securely message the owner of the server to verify it before you accept it. Details are of course left up to the user because any default system you set up will of course be exploited. It's head of line security exploits all the way down.
(I also have three hard tokens for from banks and investments, two ubikeys (both for LastPass, one in storage), and google authenticator on my iphone for anyone that offers it... so I've taken a step backward for added security.)
Which happens to also be more secure.
Hell, I store my 2FA keys in my pw manager and have it autofill those too -- a reduction in security! But there are some services that refuse to not spam your phone number if you don't set up some 2FA.