AWS’s Elastic Load Balancer is a Strangler

https://shermanonsoftware.com/2020/03/04/amazons-elastic-load-balancer-is-a-strangler/

by jeffrey-sherman • 6 years ago

112 45 6 years ago

shermanonsoftware.com

checker2• 6 years ago

And so is nginx, haproxy, API gateway, and any other layer 7 load balancer/proxy.

joana035• 6 years ago

And people were doing that even before aws existed

Edit: you can do that with iptables

checker2• 6 years ago

I didn't know that it's possible with just iptables. This sounds pretty useful. Thanks!

nickserv• 6 years ago

Everything old is new again. Except now it's on someone else's machine and you have to pay for it...

reilly30003• 6 years ago

For what its worth, ALB is a fantastic product. As a full L7 load balancer it can stand in the place of nginx, provide OIDC auth, replace API gateway especially for high volume Lambdas, and has lots of tunable logic for running diverse auto-scaled workloads. AWS is good about not breaking APIs and contracts, so the "strangler" strategy is really about accommodating existing customers. I'm not going to say Classic Load Balancers are a dog of a product or anything, its just that they pre-date VPC, and the web stack has moved on since then.

Having ALB be such a robust, turn-key product does tempt one into giving it a lot of responsibilities which definitely lends itself towards lock-in. I guess its the same with marriage: there are always tradeoffs.

I do wish it was better integrated with ECS + AppMesh + API Gateway, so it could be like a managed GetAmbassador.io. Envoy proxies are a hell of a good idea, and I think are going to be something like the React of the backend. While one can dream about a grand unified request router for the cloud, ALB continues to innovate with things like canary deployments and awesome routing rules, all well continuing to 'just work' at web scale. Is there anything else you can throw up to 50K RPS at and not have to think about it that much?

philliphaydon3• 6 years ago

One thing is that ALB to Lambda restrictions suck.

Lambdas have a 6mb response size. Api gateway has a 10mb response size. If you go ALB to Lambda. And your response is over 1mb. Your response will fail. Because the ALB team decided to introduce a random 1mb limit, not document it as a limitation, and added it to their trouble shooting notes instead.

So if you need to return a small PDF or something from a lambda via ALB. Good luck.

jcims• 6 years ago

Somebody has a war story

internalthief• 6 years ago

It is documented, at least on this page:

https://docs.aws.amazon.com/elasticloadbalancing/latest/appl...

Right at the top of the page, underneath the header "Limits"

The maximum size of the response JSON that the Lambda function can send is 1 MB.

jedberg• 6 years ago

FYI for anyone reading, the expected architecture is to return the URL to an S3 object that is your larger than 1mb response.

The reason they do this is to keep the alb running efficiently by not having to architect for random large responses.

fulafel3• 6 years ago

That's pretty damning, lots of innocent REST messages can be more than 1MB. Can you have larger bodies in chunked encoding?

CoffeeDregs• 6 years ago

Didn't know that ALBs could front Lambdas... Will be so happy to be able to be rid of APIGW...

reilly30003• 6 years ago

At the volume I was running APIGW I saved 91% in cost moving to ALB.

fosk• 6 years ago

If you need more capabilities you can also frontend your Lambdas with something like Kong[1]. ALBs and AWS API Gateway can be quite slow and constrained if performance - and a customizable feature set - are important requirements.

[1] https://github.com/Kong/kong

sandGorgon• 6 years ago

can a Envoy be the ingress replacing nginx or traefik ? this is being debated in the k3s distribution - https://github.com/rancher/k3s/issues/817

will something like https://www.getambassador.io/reference/core/ingress-controll... , support all the usecases that a nginx ingress can ? (old certificates, spdy, etc etc etc)

reilly30003• 6 years ago

https://www.envoyproxy.io/learn/front-proxy outlines the use case for an Envoy proxy being a general purpose ingress, with added routing and observability features. Honestly I can't speak competently to what Nginx and Nginx Plus/Pro are capable of these days, especially in relation to the sidecar proxy paradigm, but I know its much more than I've ever used in practice.

watermelon0• 6 years ago

It's a great product, but it still imposes limitations on header sizes. It has limitations on the size of the entire header, size of each header line, and size of the request line (i.e. URL length).

It's quite unfortunate, since this means that some use cases are limited to the classic ELB.

[1] https://docs.aws.amazon.com/elasticloadbalancing/latest/user...

greyskull• 6 years ago

Please chime in on https://github.com/aws/aws-app-mesh-roadmap/issues/37 if you haven't already

philsnow• 6 years ago

Small nit, doesn't this assume http elbs ? I didn't think you could slice traffic in any useful way if it's raw TCP.

travbrack2• 6 years ago

Right. Without reading http headers, you can only differentiate traffic by IP and/or port.

zbentley• 6 years ago

In general this is true. But routers/loadbalancers exist for many TCP protocols (even stateful ones) as well. gRPC is probably the most popular such protocol at present.

amdavidson• 6 years ago

No, ELB can terminate TLS.

https://aws.amazon.com/elasticloadbalancing/features/?nc=sn&...

crankylinuxuser• 6 years ago

My biggest gripe is that the ALBs do not handle SSL decryption and re-encryption in a safe way.

It's not my opinion that it's not safe, but FedRAMP. I'll just leave that at that.

_nalply• 6 years ago

Sounds really good. I didn't know that some people calls this a Strangler.

I have only one small caveat if your work is full of politics: The pressure to provide a replacement of the legacy system disappears and a strangler might transmute into the saving net of the legacy system.

Now you have two problems instead of one: the strangler AND the legacy system.

YawningAngel• 6 years ago

I think this is a good thing. Moving from having no choice but to replace an entire system to being able to work on it in incrementally makes an otherwise impossible problem tractable. If you end up only replacing the problematic parts of a legacy system and retaining the rest I think that's fine.

closeparen• 6 years ago

The legacy has to be really bad for an abandoned mid-flight migration to be any better. Usually the intermediate state is at least a little worse (but worth it because you eventually get through). Strangler pattern only helps bound the damage.

jerf• 6 years ago

All following the Strangler pattern can do is provide you a path forward where previously you may not have been able to even see a path. It can't force you to walk it.

ufmace• 6 years ago

I haven't had to use the Strangler technique, but I think I would disagree with this post. It is possible to use ALBs in this way, but I think it goes against the point a little. I thought the point was that it should be a little bit annoying to maintain all of the pass-through routes in the replacement codebase, to ensure that there is motivation to actually do the replacement, instead of just leaving it in place indefinitely.

Plus, leaving the pass-through in the codebase lets you do a much wider variety of things. You could run both the original and your replacement, and check the responses against each other. You could do that, or do replacements, at however tiny of a part you can imagine. You could choose which cases to handle where according to any algorithm you can dream up.

vorpalhex• 6 years ago

I've always called this technique "proxying", since most of the software that does this is referred to as "a proxy".

Thaxll• 6 years ago

Yes it's called A/B testing, weighted load balancing ect...

luord• 6 years ago

This is true for pretty much any load balancer or even any reverse proxy, I think; but nevertheless, it's true and a good way to think about stranglers.

AWS’s Elastic Load Balancer is a Strangler

Related Stories