And one can argue that if an attacker can do that, they are already inside the machine. To that effect, they could just put a bogus pass binary in ~/bin and extract all your passwords.

Yet if you use this solution (or keepass or whatever you want) you are exposing yourself to attacks to those codebases. Which normally are monstruos for the most juicy of programs: a password manager.