You could do it like a brownout, and have a random per-user cooldown going from a day to a month.

Not enough to accumulate tech debt, enough to mitigate the potential impact of any supply-chain vulnerability.