Re:We Don’t Run Cron Jobs (2016)
by _kevinspencer on Tuesday August 21, 2018 @12:54AM
(#17807114)
Not only that, but just ssh'ing in and blindly rerunning the failed job isn't the answer anyway. Research why it failed. If the job has to write a file on a full disk, you can rerun that thing a hundred times and it will never work. I'm sure they must have missed something from their write up as I can't imagine that's their on call playbook.
Re:We Don’t Run Cron Jobs (2016)
by TheSoftwareGuy on Monday August 20, 2018 @09:51PM
(#17806177)
You could even write a cron job to clean up the crontab
Or you could use at(1) or batch(1) to invoke the job once. That seems significantly easier and less error prone than trying to clean up crontab.
at(1) is great, but you'll still have the problem of a different environment to cron (it copies most of your environment variables and working directory).
Or use Puppet/Salt/Chef/ansible to manage the file automatically
Re:We Don’t Run Cron Jobs (2016)
by toomuchtodo on Monday August 20, 2018 @10:29PM
(#17806386)
I've seen this done previously. It gets arduous to maintain a large list of cron jobs, teach everyone how to add new ones, rebuild the cron box and redeploy on updates to config management.
Just use Jenkins, until you outgrow Jenkins. Then use a distributed scheduler.
Can you recommend one?
We recently had to implement a prototype in house, not that easy as I thought it would be, but we could not find a good solution, e.g. handle 100k jobs, long term (never expiring), short term (depends on an entity's live cycle), high available.
Thanks.
Re:We Don’t Run Cron Jobs (2016)
by toomuchtodo on Tuesday August 21, 2018 @09:53AM
(#17809539)
Kubernetes if you're a containers first shop, Hashicorp's Nomad if you're not just containers (can run binaries, etc).
Re:We Don’t Run Cron Jobs (2016)
by spiffytech on Tuesday August 21, 2018 @07:54AM
(#17808779)
Kubernetes does Cron jobs.
No doubt this works if it's an established procedure, but if I was approaching a system I wasn't familiar with I would never do (3) because the environment can differ wildly between crond and a login shell. It is safer to edit the cron schedule and duplicate the entry with a time set for few minutes in the future. (And clean that up afterwards).