Cylc – a workflow engine for cycling (and non-cycling) systems

by kinow • 7 years ago

12 1 7 years ago

kinow• 7 years ago

Hi!

Cylc is a GPL system written in Python, and used in some countries (Au/NZ, UK, South Africa, Korea, other institutions in Europe and US) mainly to run cyclic workflows, e.g. forecast models.

It supports the same syntax as Cron for scheduling jobs (called suites in Cylc) but with the concept of time points. So you are able to configure a workflow with some tasks that would run every midnight, other tasks that would run every 5 minutes. And in case any of these tasks fail, Cylc has a mechanism to restart the workflow and catch-up with these tasks.

There are a couple of commercial systems that provide similar features, and also Luigi, Airflow, but with limitations (mainly around cyclic tasks, and the catch-up mechanism).

There is a new web GUI under development, and questions, suggestions, or issues/pull requests welcome!

Its web site is https://cylc.github.io/cylc/ but we also just (yesterday) acquired cylc.org and will start updating the site and documentation soon.

Cheers

Cylc – a workflow engine for cycling (and non-cycling) systems

Related Stories