Re:The worst bug we faced at Antithesis
by justinsaccount on Wednesday May 22, 2024 @02:11PM
(#40444122)
I think you underestimate how badly people mess this up now, with piles and piles of cron jobs and timers all configured to run at 1/5/15/60 minute intervals.
If other things did this there would not be "more collisions", since the status quo is multiples of 5 minute intervals everywhere.
Fair point. I'll definitely consider co-primes the next time I deploy a system with complicated scheduling.
Re:The worst bug we faced at Antithesis
by justinsaccount on Wednesday May 22, 2024 @05:28PM
(#40446821)
for what it's worth, it also comes in super handy when trying to troubleshoot existing systems. Running a complicated system that has weird latency spikes sometimes at a 60s interval and the config file has like:
- timeout: 1m
- expireInterval: 1m
- updateInterval: 5m
- etc
The first thing I do is go in and change them all to - timeout: 59s
- expireInterval: 61s
- updateInterval: 307s
- etc
and then see what the new "problem interval" is. It's either gone, because the spike was from contention from all 3 jobs running every 5 minutes, or the new problem occurs now at 61s intervals.
Edit: Yeah, I guess GP said "if you control everything". Still, how often do you actually control everything? Or how long can you control everything? Everything (heh) sufficiently complex to worry about this talks with other systems at some point, right?