Of course, if lots of money is at stake, it's not uncommon for the attacker to track you down, beat you up/kill you and steal your phone (or RSA keyfob) to finish the transaction. To get the PIN that goes with the keyfob they'll use lead pipe cryptography.
Dedicated people will get what they want. Google sending you an SMS is less of a risk than a bank calling you because it's unlikely that anyone would need your Google account as much as your bank account. And on top of that, you're more likely to be kidnapped if you have an authenticator like a "something I know" that is difficult to steal.
The real point is: there is no such thing as secure. To protect your money, spread it out across multiple banking institutions with different methods of access to increase attack surface, and don't log into your savings account.