Scaling Down Gracefully with AWS Lambda and HAProxy

https://www.contentful.com/blog/2016/04/27/aws-lambda-haproxy-scaling-down/

by realityking • 10 years ago

65 14 10 years ago

andonaeus• 10 years ago

Interesting article. While my company's cloud infrastructure is in its final nascent stages, I will definitely share this article and the ideas therein for basically reloading the HAProxy context w.r.t the proxied backends.

atom_enger2• 10 years ago

How do you ensure that no requests are lost when you reload HAProxy?

jvehent• 10 years ago

Soft reload takes care of that for you, sockets aren't dropped.

If you restart, however, everything's dropped.

atom_enger2• 10 years ago

Is that new in 1.6? As of 1.5.11 there was a lot of hacking required to get a true zero downtime reload: http://engineeringblog.yelp.com/2015/04/true-zero-downtime-h...

totally• 10 years ago

So HA only updates when chef runs (every 15 mins or so)? And when nodes are deleted, I guess something else has to clean them from chef? You'll also have to add these hosts to monitoring config.

I implemented something similar with puppet/mcollective/SQS years back. Poor man's service discovery. Here are some other options that, if you can handle the complexity of running Zookeper (or similar), are probably a bit cleaner:

http://jasonwilder.com/blog/2014/02/04/service-discovery-in-...

angryasian• 10 years ago

What are the conditions that using HAProxy is preferable over using ELB or in conjunction with ?

toomuchtodo3• 10 years ago

When you want more control over your load balancing, or you have long lived connections that exceed the connection timeout you can configure with an ELB (RabbitMQ connections, for example; ask if you want deets, we've tried using ELBs multiple times with Rabbit, and always come back to HAproxy).

bmurphy1976• 10 years ago

Haproxy also supports mutual SSL and can do content-based health checks. Logs don't have to go through an S3 bucket so you can also get real-time metrics and logging if you want.

Also, because you have control over it, you have more options for service discovery (i.e. write a Python script that periodically queries a Kubernetes cluster).

And virtual hosts... And SNI... And about a thousand other features.

Honestly, ELBs are pretty crappy.

whalesalad• 10 years ago

We had issues with internal ELB's for RabbitMQ (the 1 hour idle connection timeout) and the solution ended up being a heartbeat in the clients. Most client libraries have a parameter that will perform a heartbeat at some interval to keep the connection open.

I can't imagine deploying our own HAProxy infrastructure. Sure, it's relatively simple, but the set-it-and-forget-it comfort provided by ELB (not to mention free SSL) is hard to beat.

Scaling Down Gracefully with AWS Lambda and HAProxy

Related Stories