wouldn't it have made it harder to find in this case if the DCHP lease time was 34 minutes?
If it's in your control use co-prime scheduling. If it's not in your control, um... Hope they didn't use co-primes? Or a multiple of your co-primes? Er, yeah. I see the justification for doing it, but it's not exactly a cure for complexity. It'll work up until everyone else catches up with your weird trick, and then there might be may be more collisions than there were before.
Edit: Yeah, I guess GP said "if you control everything". Still, how often do you actually control everything? Or how long can you control everything? Everything (heh) sufficiently complex to worry about this talks with other systems at some point, right?
Re:The worst bug we faced at Antithesis
by justinsaccount on Wednesday May 22, 2024 @02:11PM
(#40444122)
I think you underestimate how badly people mess this up now, with piles and piles of cron jobs and timers all configured to run at 1/5/15/60 minute intervals.
If other things did this there would not be "more collisions", since the status quo is multiples of 5 minute intervals everywhere.
Fair point. I'll definitely consider co-primes the next time I deploy a system with complicated scheduling.
Re:The worst bug we faced at Antithesis
by justinsaccount on Wednesday May 22, 2024 @05:28PM
(#40446821)
for what it's worth, it also comes in super handy when trying to troubleshoot existing systems. Running a complicated system that has weird latency spikes sometimes at a 60s interval and the config file has like:
- timeout: 1m
- expireInterval: 1m
- updateInterval: 5m
- etc
The first thing I do is go in and change them all to - timeout: 59s
- expireInterval: 61s
- updateInterval: 307s
- etc
and then see what the new "problem interval" is. It's either gone, because the spike was from contention from all 3 jobs running every 5 minutes, or the new problem occurs now at 61s intervals.
Re:The worst bug we faced at Antithesis
by justinsaccount on Wednesday May 22, 2024 @02:31PM
(#40444400)
Why?
At some point you'll have a weird problem, or a load spike that shows up at regular intervals. If all of your intervals are 5/15/30 minutes, you will have 2 things running every 15 minutes and 3 things running every 30 minutes, you won't necessarily know which one causes the issue.
If you use (co)prime numbers, say, 5/7/11/13/17/19 as intervals: One, you won't have a thundering herd of tasks all running at the exact same time every few minutes, and two, when someone notices a weird issue that happens every 17 minutes, you will know exactly what the cause is.