Password managers (or a GPG-encrypted file) work just fine. Yes, you're in trouble if you forget the password, so don't do that. (If you don't have regular backups and worry about forgetting the password, fix the backups issue first - drives die more often than you'll forget a password.)