I'm not sure we are talking about the same thing, because I see no need at all to backfill monitoring data. In fact, it's one of my guidelines to decide if something is monitoring or logging; logs can not have holes.

You push as a best effort. It's up to the receiving party to react to a lack of data.

> Pushing has the Byzantine generals

not really.

the general's problem is about trust and authentication.

however, to your point about not being sure if the metrics ever get there, there is another way.

If you just have metrics, then its a single point of failure. so you need another basic "alive" check. for web services most people already have some sort of /__health or other check to allow loadbalancers to work. marrying up the two sources of data allows you to work out if its the service, metrics or loadbalancer thats not working.