That didn't happen in the past and I doubt it will be true in the future; in fact I'd say one of the reasons ARM remained competitive is because of conditional execution, the "free" barrel shifter, and Thumb mode, which really help with code density (directly related to cache usage) and avoiding branches.
AArch64 looks very much like MIPS, an ISA that hasn't really been known for anything other than being cheap and a good simple pedagogical aid (despite plenty of people being convinced it would easily outperform x86 at a fraction of the cost.) I'd guess any perceived performance increases over AArch32 are primarily due to the widening to 64 bits, and in any case much of the benchmark performance rests on the special functional units (SIMD, crypto, etc.)
No compiler developer would agree with you. The conditional execution wreaks havoc with dependencies, and branches are very cheap if correctly predicted. The barrel shifter is not as useful as you would think (what fraction of instructions are shl or shr?)
Thumb mode does help code density, but not as much as you might think due to Thumb-1 not being practical and Thumb-2 being fairly large. AArch64 is quite a bit denser than x86-64 already.
It is true that the ISA doesn't matter too much from a performance point of view. But why not take advantage of the necessary compatibility break to clean things up? There's a lot of needless complexity in our ISAs from the programmer's point of view, and cleaning it up is just good engineering practice. Let's not saddle future generations with the mistakes of the 1980s.
Personally, I prefer ARM's move, because while it's a complexity increase now, it paves the way to someday drop support for 32-bit ARM, which would be a major architectural simplification. It also means that, as a programmer, when you're in 64-bit mode you aren't burdened by all of that weird backwards compatibility stuff going back decades—you get a relatively clean ISA.
(I do have some gripes with AArch64, to be fair: there are too many addressing modes and the condition codes are unnecessary. But I'll take anything that moves us in a more RISC-y direction.)