I would say the trust issues with build remotes and substitutors are fairly independent of the caching policy?

The caching model is about when the drv changes ((drv, output) is the cache key, basically), the trust model is one one can download something to avoiding build to fill in the value for the new cache key.

Note also that with the new floating content-addressed derivations, the trust stuff is much easier to think about.