Leaks can be client-side, too. Outdated or zero-day exploits could easily allow attackers to get a (replayable!) password or hash from a browser, improperly terminated VPN, SSL stream, etc.

An ephemeral TOTP value is almost useless to them in this case.

As for rubber hose: if your 2FA smartcard/token/device isn't carried after you leave the office (for example), attackers getting your password via a mugging out the street is less useful.