There are ways to mitigate, such as multisig wallet. For day to day, use a wallet with a small amount. When it's balance runs low, you can replenish the amount from your vault, that requires at least 2 signatures. Crypto is not about completely eliminating trust from the system, but rather being able to choose whom you trust and control, what a trusted party can do.