Dalga: Job scheduler written in Go

https://github.com/cenkalti/dalga

by cenkalti • 11 years ago

56 14 11 years ago

github.com

rbanffy• 11 years ago

Why Dalga instead of plain cron?

cenkalti3• 11 years ago

* Dalga stores jobs in a MySQL table which is handy if you are already using MySQL (backups, etc.)

* Dalga can schedule one-off jobs.

* Dalga has built-in retry mechanism.

jerf• 11 years ago

A tight dependency on MySQL is somewhat limiting. Have you considered making that an interface with a default implementation that uses MySQL? It pretty much ought to be a mechanical transform where you simply extract all the places you touch MySQL into a method, then write an interface around that method.

jlouis• 11 years ago

In large setups, it is usually clever to isolate job execution policy into a separate system. This simplifies a lot of contextual configuration of the system at large.

You can have cron emit the cascading event if you want but you often need a system which can "prod" other systems and ask them to do things because of a timing event. In such systems, cron often falls short as the system around it grows.

For local timing of events, cron is a really good tool however, and I wouldn't hesitate about using it there. It is for system-wide events these general job schedulers tend to help.

x0rg2• 11 years ago

Why Dalga instead of Kala? https://news.ycombinator.com/item?id=10046164 Aren't they very similar? Isn't it better to join the efforts?

cenkalti3• 11 years ago

Kala is in Alpha stage. Dalga is stable and used in production ~2 years at https://put.io.

x0rg2• 11 years ago

So I should probably ask the same question in the other thread :)

coverband• 11 years ago

On a completely different note, kudos for the idea of charging a small amount that gets contributed to charity for trial accounts (on https://put.io/).

Not sure if you guys came up with that, but it's the first instance I've come across. Great idea -- filters out freeloaders from potential true customers.

rkrzr• 11 years ago

Does Dalga support job dependencies? I.e. job A can only run after e.g. job B and C have finished first.

cenkalti3• 11 years ago

No.

mystique• 11 years ago

Have you looked at Rundeck? http://rundeck.org/

It has rest based interface to do everything + UI to make it easier when you need to do one off things. It allows you to control execution across multiple nodes. Good thing is you can setup your processes by tag of a node in Dev and export the project as is on Production and it will still work.

It's a runbook automation tool but can be used like in place of chronos, oozie, spark job server etc.

jksmith2• 11 years ago

Why Dalga instead of of some other job scheduler? Because all service software will eventually be rewritten in golang, apparently ;). Don't hate the player.

wilsonfiifi• 11 years ago

To be honest, I like the fact that a lot of the 'old` stuff is being rewritten in Go because I find it a much easier language to grasp than C, C++ or Java. So I say rewrite them all! (just my opinion of course)

jksmith2• 11 years ago

Well that's a driver for golang for sure. Additionally, while we're complaining about golang not measuring up to our programming sensibilities, the people who write us checks want 1) correct, 2) on time, and 3)in budget. If golang helps us do that better than other languages in spite of the fact that it doesn't have generics or whatever (anecdotally seems to be the case), no need to apologize for it.

Dalga: Job scheduler written in Go

Related Stories