Load balancing at Github: Why ldirectord?

http://www.anchor.com.au/blog/2009/10/load-balancing-at-github-why-ldirectord/

by fogus • 17 years ago

27 5 17 years ago

anchor.com.au

"I don’t think queueing something as time-sensitive as interactive HTTP requests is a good idea."

Queuing will happen somewhere in your architecture regardless. If I understand github's setup properly, it'll be in the operating system's accept queue for each backend. We know from queue theory that a queue per worker is inferior to a single queue for all workers.

jeremyw• 17 years ago

Yeah, Matt lost me here (in his reply to Willy Tarreau's [HAProxy] comment.) Packet rewriting is nice and efficient, but it's dumb. Unless you measure a latency/cpu issue with proxies, in a system with lots of moving parts, having high-level ways to control the flow of traffic -- on-the-fly! -- is a good thing.

adrinavarro• 17 years ago

I once worked with ldirectord (I was shown at a local yet huge startup how they work) and I can say it's pretty awesome. You may integrate what it does at a router-level (newer top-class Cisco routers with modified IOS can do client-persistent –if needed– packet balancing, without having to forward also the responses)

It's pretty awesome to just redirect incoming (usually small) packets and let indivual machines (placed in other farms, if needed) send the response without worrying about it.

egonzalez• 17 years ago

If you are interested in ldirectord take a look at the ultramonkey project. http://www.ultramonkey.org/ I've used this on very high traffic sites () on p4 servers. It's almost a replacement for the f5 BigIp boxes.

curse.com survived patch days. * mainstream media site +600mbs

jbyers• 17 years ago

"If all you needed was a HA load balancer, and had no experience with either ldirectord or keepalived, I’d probably recommend keepalived over ldirectord, as it’s one project and one piece of software to do everything you need."

Having used both ldirectord and keepalived at reasonable scale this strikes me as solid advice. We use keepalived -- formerly to run LVS, now has a pure IP failover solution for nginx -- and it's been a rock for years. ldirectord is great for the LVS side too, but having one less moving part to worry about is a nice plus. Of course, going with what you know trumps as this review states right up front.

I don't agree about the overhead of haproxy and proxies and general. The author's assertion is that proxies are slower than LVS in high traffic situations. He's certainly right, but any performance loss is vanishingly small on modern hardware. We have redundant nginx pairs up front (tens of millions of requests per day, 100 Mbps+) on modest servers that are nearly idle. If there's a difference between this setup and its LVS predecessor, it's too small to measure. In exchange, we get huge flexibility in configuration, fewer moving parts, and end-to-end logging. If we get up to something in the Gbps+ range we would probably LVS up front again, but until then, I'm pretty happy with my proxies.

Load balancing at Github: Why ldirectord?

Related Stories