Murder: Fast datacenter code deploys using BitTorrent

http://engineering.twitter.com/2010/07/murder-fast-datacenter-code-deploys.html

by abraham • 16 years ago

63 26 16 years ago

engineering.twitter.com

maukdaddy2• 16 years ago

  no NAT/Firewall issues, no ISP traffic shaping, only trusted peers

Sounds like an easy-to-exploit datacenter to me.

winthrowe• 16 years ago

They'd have those protecting the perimenter, but this is only for deploys in a single datacenter right now. They're only doing it on their network, so they don't have to worry about all the hoops torrents normally go through; they can drop the timeouts to nothing, they don't need encryption, etc.

The video states that they will be looking into how best to do deploys across centers in the future, but they are not going to have peers connect across their wan links. One tracker and seeder per location is a first musing.

[Edit: paragraphs]

koenigdavidmj• 16 years ago

>The video states that they will be looking into how best to do deploys across centers in the future, but they are not going to have peers connect across their wan links. One tracker and seeder per location is a first musing.

Heck, use a VPN to accomplish this if you're lazy.

naner• 16 years ago

Sometimes I wish geeks wouldn't worry about giving something a cool sounding name and instead just picked something conventional or unremarkable.

I don't want to search Google for help with Murder. I don't want to discuss Murder at the office.

maukdaddy2• 16 years ago

or the next morning....did you complete the murder job?

samratjp• 16 years ago

or commit murder (on github obviously)

fizx• 16 years ago

Don't worry. Murder only takes a few minutes.

joubert• 16 years ago

The user manual can be called Murder, She Wrote.

derwiki2• 16 years ago

At least it's not Mrdr.

DaemonXI• 16 years ago

One does not simply Bittorrent into Mrdr.

jcapote• 16 years ago

You guys know a group of crows is called a Murder, right?

I'd hate to work somewhere where I could only use stuffy, politically correct software names. Call me crazy, but quirky software names are half the reason I release a project.

jolan• 16 years ago

Indeed. I called my asset manager 'assman'. :)

mrduncan• 16 years ago

Context: http://www.youtube.com/watch?v=tujqM2u-BVo

mhd• 16 years ago

It's a neat idea to use collective nouns at a bird-themed company, there's only one problem: crows aren't known for their tweeting. Crows, well, crow.

Flight, flock, bevy would be pretty generic, pick a songbird and you'd get something more interesting.

gxti• 16 years ago

As long as it doesn't end in -ly or -ing, I'm pretty much happy.

Jim_Neath• 16 years ago

I remember the uproar when I released a plugin called Shemail.

alec2• 16 years ago

I wonder if they tried multicast. It would be strictly more efficient in terms of bandwidth (one copy per router/hop rather than all the different pieces flying back and forth) and likely less CPU. There's premade software like uftp (http://www.tcnj.edu/~bush/uftp.html) to do this.

moe• 16 years ago

Amen. The NIH syndrome at twitter seems to be going strong. They just don't stop reinventing stuff, and usually poorly.

In this case uftp is the time-tested solution, made precisely for this purpose (mass deployment) and provably better on all metrics (speed, bandwidth, security via SSL, complexity).

The mind remains boggled...

nullstyle• 16 years ago

You do realize that twitter didn't invent bittorrent, right?

;)

DannoHung• 16 years ago

Maybe this is too easy, but if you're using Git, why not have servers that get a new release be added to a list of servers that can distribute a new release?

derwiki2• 16 years ago

Added complexity to continue to scale, and one deploy replica high up in the chain that breaks will break everything downstream. As you start building a system to address those issues, you're adding more complexity. This would lead me toward a Murder based solution from the start.

bonzoesc• 16 years ago

Couldn't they use a tree-like distribution system where each node only receives pulls from a small number of child nodes?

You could organize this by giving each server an incrementing number and using math to deduce their parent node, much like an implicit binary tree stored as an array: http://en.wikipedia.org/wiki/Binary_tree#Ahnentafel_list

That way you can still use the http or git protocol without having to invent a special flavor of BitTorrent.

bjg• 16 years ago

He covered why they didn't want to use a tree like distribution method in his presentation.

It introduces a possible single point of failure for all nodes which are children of a failed node.

metamemetics• 16 years ago

css text-shadow on body content is over-rated and actually makes it harder to read.

Murder: Fast datacenter code deploys using BitTorrent

Related Stories