The same argument could be made about online banking or any other high-risk online service. Really though, your average manager far more likely to have their machine compromised with malware or fall for a spear phishing attack. There will always be ways to circumvent security, but password managers solve more problems than they create. Regardless if it was your password manager which was compromised, or any other sort of password leak, it's really easy to quickly change all of your passwords once it's secured again. Eschewing good because it's not perfect is a losing strategy.