There's a certain level of truth to it, as none of the Android phone makers appear to be able to push for a customized SoC, even the ones with own designs (Samsung) appear to keep a "wall" between mobile and SoC divisions.
This means that relatively easy wins like throwing a lot more cache on high end phone, are unavailable because instead they are just one of the many customers for the SoC with different priorities resulting in a compromise design.
Is it ? I'm using a 4 years old $200 phone which suits me just fine, if not for stupid apps requiring more resources for a worst experience. The only reason of upgrade is battery wearing off.
Any Google related issues might be mitigated switching to LineageOS as the hardware is relatively open.