Kala: A Modern Job Scheduler (written in Go)

by jaytaylor • 11 years ago

64 31 11 years ago

endymi0n• 11 years ago

Exactly what I was looking for since months now!

Loved the architecture of Chronos, but with JVM, Zookeper and Mesos as requirements, it had "heavyweight" written all over the place. This is what the successor of cron should be IMHO. Distributed by design, API driven and dynamic.

EDIT: Apparently it's NOT distributed yet. Had hoped for Raft or etcd integration.

samdolan8055• 11 years ago

Feel free to throw issues into the GH tracker so we can get them on the roadmap.

My initial use case for this is for complicated time-based triggers that would be an absolute PITA/impossible to model in cron. We both felt the pain of setting up celery, etc.; @ajvb took that pain and put this together.

thwarted• 11 years ago

https://github.com/erikh/jobd uses etcd for distribution of jobs.

ajvb5• 11 years ago

The initial use case was for a single node setup, Chronos for an early stage startup. I/we will be adding support for it to be distributed here soon, but not until the single-node, single binary with no external requirements setup is production ready.

If you have any feature requests, please create an issue in github. The more people request something the sooner it will get implemented.

Thanks for the feedback though.

aukhan• 11 years ago

is there way to migrate existing jobs from tools like jenkins to kala ?

atmosx2• 11 years ago

I'm sure it's possible, but it's marked as 'Alpha' in bold letters, "not production ready".

samdolan8055• 11 years ago

Can you start an issue so we can start discussing what this would look like? What kind of APIs would you need for this?

dmix2• 11 years ago

This looks good, nice work.

I need something like this at the moment, but I'm curious what the target company size is for a project like this. For example, we're a 2-3 person team and not quite big enough to have the capacity to run our own deployment of something as critical as Kala/job schedulers. Nor do I want to run another DB like Bolt/Redis when I'm already running another critical DB.

I'll probably go with a hosted solution ala http://www.iron.io/worker until we can run our own.

shazow• 11 years ago

Bolt is an in-process database, like sqlite or leveldb. And it's native-Go, so there are no dependencies. You don't need to run additional service infrastructure to use it.

carimura• 11 years ago

Just for clarification, the IronWorker platform by Iron.io supports large enterprise deployments in cloud, on-premises, and hybrid environments, so it scales from small hosted projects to very large deployments processing millions of jobs per hour. http://www.iron.io/enterprise/

ajvb5• 11 years ago

dmix, a 2-3 person team is actually exactly what this project was built for. The use case came from a start-up of the past that I was an engineer at that really really needed something like this.

dmix2• 11 years ago

Fair enough, different teams have more resources for sys admin work or are at a later stage where they can invest in it. I'm just wary of adding any external pieces of software after being burned a few times staying up late on weekends fixing a broken service.

guruparan18• 11 years ago

Side note: "Kala" means time, in Sanskrit. It also means someone who controls "death". Feminine form of it is "kali". I think it is only fitting to name a scheduler after time (or something that is controlled by it). ;)

[1] https://en.wikipedia.org/wiki/K%C4%81la_(time)

deanclatworthy• 11 years ago

Sadly in Finnish it means "fish".

atmosx2• 11 years ago

Luckily in Greek it means 'good'.

ajvb5• 11 years ago

It was taken from the Sanskrit meaning for "time". :)

izyda2• 11 years ago

Looks good. Very much looking forward to this being production ready. I am currently looking for a scheduler that provides a better UI/UX than cron alone.

Currently, what do you all use for this sort of thing?

Nextdoor has a not-yet-open-sourced solution: https://engblog.nextdoor.com/2015/06/10/we-do-not-run-cron-j... that looks great.

Does anyone have any experience with Yelp's tron? https://github.com/Yelp/Tron

samdolan8055• 11 years ago

I work with @ajvb. We'll be working on getting this into production over the next 2 months. To do so, we'll be working on adding redis as a backend and making it distributable. The goal is to make it really easy to get started, but be able to scale out to bigger production systems easily.

@ajvb has plans for an awesome dashboard as well to complement this :) Stay tuned... good things ahead.

izyda2• 11 years ago

Is this associated with http://www.222labs.com ?

If not, is there an email or somehow I can PM you or @ajvb at?

no1youknowz• 11 years ago

Please add postgres support.

ajvb5• 11 years ago

I had not seen Nextdoor's scheduler, which looks awesome except for the required external requirement of SQS. Kala was built to not require anything except for the Kala binary.

Yelp's Tron is awesome, but I wanted an JSON over HTTP API and ISO 8601 Duration Notation. Plus there are a few over problems with it that I currently don't remember.

Kala: A Modern Job Scheduler (written in Go)

Related Stories