No, some DL architectures are unquestionably superior to others especially for some tasks. In the first few years of deep CNN development, several limits quickly became apparent (sensitivity to scale, vanishing gradients, gradient signal loss with some activation functions, and many engineering constraint tradeoffs like poor RAM use). This was addressed by making changes to a simple dense net which begat architectures that suffer less from those limits.

Without question, limits exist in every DL architecture; we just haven't taken the time to diagnose or quantify all the limits yet. Now that attention has shifted to transformers, analysis of DNNs' inherent limits is made substantially more difficult, given transformers' huge size. That, and their multi-million dollar training cost likely will make if infeasible to diagnose or cure each design's inherent limits.