I believe the Apollo Guidance Computer used a hardwired lower segment that always points to Bank 0 and one for Bank 1. Then a few freely changeable segments.

I also recall some systems where it works just as described, but you had to be incredibly careful about bank switching and IIRC there was a dedicated instruction to write the bank register and program counter register at the same clock cycle.