The thing I keep in mind is that state is mutated far more than it is observed. You might handle 1000 or more requests per second and only export the number of requests once per second or less. In light of this ratio it’s important to make the recording path as simple and cheap as possible, and it’s ok if the observing path has to be complicated to compensate.

I usually refuse to use atomic increments for services because it scales very poorly. Even a mutex-protected increment scales much better than atomic increment.