There have been kerfluffles in the past where people (companies) did exactly that. Gave someone a binary, then told them to download the source from somebody else. People got angry.

Yes. But Debian wouldn't be angry if you pointed to them.

In the past, it's been business entities that have distributed considerable a number of binaries (I believe one case was with code running on a router?) -- and charged quite a bit for that (bundle) -- and yet been too cheap to host a copy of the code online.

Usually those companies were shipping modified versions, and/or it wasn't possible to build your own modifications for the device. I don't recall an instance - though it may well just be that I'm ill informed - where "people got angry" just because a company wasn't mirroring something.