Re:Kala: A Modern Job Scheduler (written in Go)
by endymi0n on Wednesday August 12, 2015 @03:25AM
(#10046534)
Exactly what I was looking for since months now!
Loved the architecture of Chronos, but with JVM, Zookeper and Mesos as requirements, it had "heavyweight" written all over the place. This is what the successor of cron should be IMHO. Distributed by design, API driven and dynamic.
EDIT: Apparently it's NOT distributed yet. Had hoped for Raft or etcd integration.
Re:Kala: A Modern Job Scheduler (written in Go)
by ajvb on Wednesday August 12, 2015 @06:07PM
(#10051019)
The initial use case was for a single node setup, Chronos for an early stage startup. I/we will be adding support for it to be distributed here soon, but not until the single-node, single binary with no external requirements setup is production ready.
If you have any feature requests, please create an issue in github. The more people request something the sooner it will get implemented.
Re:Kala: A Modern Job Scheduler (written in Go)
by samdolan805 on Wednesday August 12, 2015 @09:49AM
(#10047648)
Feel free to throw issues into the GH tracker so we can get them on the roadmap.
My initial use case for this is for complicated time-based triggers that would be an absolute PITA/impossible to model in cron. We both felt the pain of setting up celery, etc.; @ajvb took that pain and put this together.
Re:Kala: A Modern Job Scheduler (written in Go)
by guruparan18 on Wednesday August 12, 2015 @04:57AM
(#10046807)
Side note: "Kala" means time, in Sanskrit. It also means someone who controls "death". Feminine form of it is "kali". I think it is only fitting to name a scheduler after time (or something that is controlled by it). ;)
Re:Kala: A Modern Job Scheduler (written in Go)
by izyda on Wednesday August 12, 2015 @08:36AM
(#10047340)
Looks good. Very much looking forward to this being production ready. I am currently looking for a scheduler that provides a better UI/UX than cron alone.
Currently, what do you all use for this sort of thing?
Re:Kala: A Modern Job Scheduler (written in Go)
by samdolan805 on Wednesday August 12, 2015 @08:47AM
(#10047387)
I work with @ajvb. We'll be working on getting this into production over the next 2 months. To do so, we'll be working on adding redis as a backend and making it distributable. The goal is to make it really easy to get started, but be able to scale out to bigger production systems easily.
@ajvb has plans for an awesome dashboard as well to complement this :) Stay tuned... good things ahead.
Re:Kala: A Modern Job Scheduler (written in Go)
by samdolan805 on Wednesday August 12, 2015 @09:39AM
(#10047607)
We work together there. We actually both started working on a job scheduler independently, but joined forces and he ran with it. It's @ajvb's personal project, but we'll be using it at 222 labs for sure. You can shoot me an email at my username without the numbers @ gmail.com and I'll hook in @ajvb to the thread.
Re:Kala: A Modern Job Scheduler (written in Go)
by ajvb on Wednesday August 12, 2015 @06:14PM
(#10051069)
I had not seen Nextdoor's scheduler, which looks awesome except for the required external requirement of SQS. Kala was built to not require anything except for the Kala binary.
Yelp's Tron is awesome, but I wanted an JSON over HTTP API and ISO 8601 Duration Notation. Plus there are a few over problems with it that I currently don't remember.
Re:Kala: A Modern Job Scheduler (written in Go)
by dmix on Wednesday August 12, 2015 @04:20AM
(#10046702)
This looks good, nice work.
I need something like this at the moment, but I'm curious what the target company size is for a project like this. For example, we're a 2-3 person team and not quite big enough to have the capacity to run our own deployment of something as critical as Kala/job schedulers. Nor do I want to run another DB like Bolt/Redis when I'm already running another critical DB.
Re:Kala: A Modern Job Scheduler (written in Go)
by shazow on Wednesday August 12, 2015 @04:45AM
(#10046781)
Bolt is an in-process database, like sqlite or leveldb. And it's native-Go, so there are no dependencies. You don't need to run additional service infrastructure to use it.
Re:Kala: A Modern Job Scheduler (written in Go)
by carimura on Wednesday August 12, 2015 @11:04AM
(#10048066)
Just for clarification, the IronWorker platform by Iron.io supports large enterprise deployments in cloud, on-premises, and hybrid environments, so it scales from small hosted projects to very large deployments processing millions of jobs per hour. http://www.iron.io/enterprise/
Re:Kala: A Modern Job Scheduler (written in Go)
by ajvb on Wednesday August 12, 2015 @06:16PM
(#10051090)
dmix, a 2-3 person team is actually exactly what this project was built for. The use case came from a start-up of the past that I was an engineer at that really really needed something like this.
Re:Kala: A Modern Job Scheduler (written in Go)
by dmix on Thursday August 13, 2015 @08:19PM
(#10057971)
Fair enough, different teams have more resources for sys admin work or are at a later stage where they can invest in it. I'm just wary of adding any external pieces of software after being burned a few times staying up late on weekends fixing a broken service.
Loved the architecture of Chronos, but with JVM, Zookeper and Mesos as requirements, it had "heavyweight" written all over the place. This is what the successor of cron should be IMHO. Distributed by design, API driven and dynamic.
EDIT: Apparently it's NOT distributed yet. Had hoped for Raft or etcd integration.
If you have any feature requests, please create an issue in github. The more people request something the sooner it will get implemented.
Thanks for the feedback though.
My initial use case for this is for complicated time-based triggers that would be an absolute PITA/impossible to model in cron. We both felt the pain of setting up celery, etc.; @ajvb took that pain and put this together.
[1] https://en.wikipedia.org/wiki/K%C4%81la_(time)
https://en.wikipedia.org/wiki/Kalā
:)
If black it's kalaa
If time it's kaa-La
Currently, what do you all use for this sort of thing?
Nextdoor has a not-yet-open-sourced solution: https://engblog.nextdoor.com/2015/06/10/we-do-not-run-cron-j... that looks great.
Does anyone have any experience with Yelp's tron? https://github.com/Yelp/Tron
@ajvb has plans for an awesome dashboard as well to complement this :) Stay tuned... good things ahead.
If not, is there an email or somehow I can PM you or @ajvb at?
Yelp's Tron is awesome, but I wanted an JSON over HTTP API and ISO 8601 Duration Notation. Plus there are a few over problems with it that I currently don't remember.
I need something like this at the moment, but I'm curious what the target company size is for a project like this. For example, we're a 2-3 person team and not quite big enough to have the capacity to run our own deployment of something as critical as Kala/job schedulers. Nor do I want to run another DB like Bolt/Redis when I'm already running another critical DB.
I'll probably go with a hosted solution ala http://www.iron.io/worker until we can run our own.