> Wouldn't it be relatively trivial for someone to compile, compare checksums and call them out?

Generally not. Most software does not have reproducible builds, so the checksums would be unlikely to match.