Speed up your Fabric deployments with Parex

http://paulbohm.com/parex-parallel-execution-for-python-and-fabric/

by enki • 16 years ago

19 8 16 years ago

njharman• 16 years ago

Why isn't Fabric doing this? This would be basic feature of deployment software I would think. I vaguely remember the old, old Fabric (before they rewrote everything) did this.

enki3• 16 years ago

i think they've been planning to for a while: http://code.fabfile.org/issues/show/19

i haven't really followed their progress too closely, but they seem to be close to releasing something for 1.0.

I think they do client-side threading while i prefer to to do fork() on the server to avoid multiple ssh connections and retain maximum control.

i'll reevaluate when they ship. till then parex works for me.

goosemo3• 16 years ago

The branch actually does client side forks, one for each server in a task. Then each call inside a task is run sequentially, as expected in normal fabric usage. I've also included the ability to set a pool size, so that one can manage the number of ssh connections open simultaneously.

jacobian• 16 years ago

There's a branch with this feature: http://github.com/goosemo/fabric. The current work's in the "mutliprocessing" branch. I'm using it for a big deploy right now, and it appears to be working well.

[edit: fixed my mistake about where the most current code lives]

goosemo3• 16 years ago

Just to note/ask though, this requires that you have a python script on the server that uses this as a lib? and requires tornado?

enki3• 16 years ago

i just use the same fabfile on the server. and doesn't require tornado on linux (if epoll is available)

Speed up your Fabric deployments with Parex

Related Stories