News for hackers. Stuff that matters.
Re:Git scraping: track changes over time by scraping to a Gi... by great_psy on Thursday August 10, 2023 @09:26PM (#37084003)
‘’’ It runs on a schedule at 6, 26 and 46 minutes past the hour—I like to offset my cron times like this since I assume that the majority of crons run exactly on the hour, so running not-on-the-hour feels polite. ‘’’

Not sure how much of a difference it makes to the underlying service but I will also do this with my scraping.

Thank you for point that out

Re:Git scraping: track changes over time by scraping to a Gi... by madphilosopher on Friday August 11, 2023 @01:24AM (#37085381)
Further to this, I also put the following snippet in front of my web-scraping cron jobs so they start at a random time between the minute boundaries:

    perl -le 'sleep rand 60';
It's the most compact code I could find to do the job.
Re:Git scraping: track changes over time by scraping to a Gi... by drougge on Saturday August 12, 2023 @10:16AM (#37100434)
If you're using a somewhat modern shell there is $RANDOM which gives you a 15 bit random number. So e.g.

    sleep $((RANDOM / 546))
but I guess most cron jobs run with an extremely conservative shell that might not have it.