I agree with you here, unless you've vetted that GPG public key very well .... it is indeed no better than trusting the CA.

In a way, having JavaScript client-side verification of files as an option would be as secure (if not more secure) in most circumstances because it'd be more noob friendly. At the very least to ensure mirrors aren't doing anything nefarious.