You can interact with legitimate financial institutions in the normal way under your public identity. You've just got to make sure you don't share that identity with your bitcoin counterparties, which will limit your ability to exchange bitcoin for things of value such as fiat currency.

Bitcoin isn’t meant to interface with existing financial institutions.

Correct. It's actually in the first line of the Bitcoin whitepaper's abstract:

> A purely peer-to-peer version of electronic cash would allow online payments to be sent directly from one party to another without going through a financial institution.

Sure, but effectively 100% of the world's economy takes place via fiat currencies. One could remain anonymous by accumulating a bunch of cash (or gold, or some commodity, etc) and never spending it as well.