But for, say, an app for a restaurant or a document reader, you would not know or have any reason to trust the vendor, so if they are self-hosting their own repos you are taking a tremendous risk trusting them.
The end result would probably remain the same - users might use third party repos for huge popular apps, but small apps would still need to stay centralized because there is no way to introduce a viable trust model against an organization you never interacted with before.
Most people don't change their default browser, adding third party repos would be similar. Removing the ability for the owner of a device to install software they want fixes one symptom, not the main issue of trust. Also it makes your device into a glorified feature phone. No thanks.
The alternative is download static apks today and maintain updates yourself(bad) or remove the freedom to install what you want on your device.