[1]: https://en.m.wikipedia.org/wiki/Historical_nuclear_weapons_s...
At a certain point bigger yields are less desirable than more, smaller yields. Picture an enormous spherical fireball, then picture chopping it up and using the "useless" slices that were at high/negative altitude to instead cover more ground. Once it became clear that they were past the regime of yield scaling, the Americans moved on to developing smaller warheads that could do just that: pepper targets and deliver more devastation per pound than a corresponding single warhead of high yield.
Vanity metrics are always more important to the second place player, and that's what Tsar Bomba was.
The USSR's followup actions give it away, though: they stopped pursuing higher yields, even though there was no theoretical or engineering limit prohibiting them, and instead chased the USA down the path of miniaturization. SLBMs and MIRVs were the future and everyone knew it.
I'd call that an engineering limit.
The “overcompensation” framing may be true, it may be a bias I’ve picked up from my society, or both or even downright false. In any case, the Soviets established a way to build and demonstrate a weapon that established the threat they wanted. And maybe the CCP will do similar, and get what they want from a path that we in the West will turn up our noses at—but that will still be effective.
The Soviet Union had managed to surpass the US in constructing nuclear bombs in a relatively short matter of time (I'd say that by the 1960s they were already in front), even though they were not as "open" and they didn't have as many Nobel laureates working for them as the US had.