I agree the resulting binaries will be different but I disagree that these are then different packages. The package is whatever the distributor distributed it as. They distributed it as source, which you took and compiled to your local binary version. With "The individual packages need to be verified", and each resulting different binary being uniquely a different package, this means the package/repo maintainer of a source package would need to verify it for every single platform which compiles C and every potential compiler and every potential set of flags for the compiler as each set of platform/compiler/flags could result in a different binary and thus different behavior.

Yeah, if I were setting up a verification service there's no way I would verify source code instead of compiled binaries.