The inlets are supposed to close before the engine enter pure-rocket mode. So I suppose that's not much more drag, especially since at this point the aircraft is at very high altitude thus with low air pressure.

So the main drawback is not drag but rather weight. Presumably, they've made the necessary calculations and the advantages of breathing air appear to make up for the additional weight.