No. The anti-proton is 1836x as massive as an electron. This makes its wavelength very short, so its lowest orbit very close to the nucleus, close enough that it has non-negligible amplitude to interact directly with a nuclear proton.

Also, its higher-orbital energy levels are very close to one another, so it is easy for it to spit out a very low-energy photon and drop to the next orbit down.