> If the government funds contractors to e.g. clean government buildings, should the contractor automatically be subject to all sorts of considerations "in the public interest" just because it is being paid with tax dollars?

IANAL, but that's actually true. For example, government contractors are required to have affirmative action programs. See, for example here: https://www.shrm.org/resourcesandtools/tools-and-samples/hr-...