Under the Hood of the Amazon EC2 Container Service

http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html

by signaler • 11 years ago

90 23 11 years ago

gtaylor3• 11 years ago

I've been doing a lot of playing with Google Container Engine (hosted Kubernetes) on Google Cloud, and have been very impressed so far. Can anyone who has worked with both offer some thoughts on how the two compare?

I don't know if Amazon's Container Service has substantial parts that have been open sourced, but I've been super happy with how Kubernetes is being developed in the open. It's also nice that it runs just about everywhere (in theory). It's pretty rough to set up manually right now, but I assume that will improve with time.

nodesocket• 11 years ago

+1 for Google Cloud (GCE). Google is really putting in effort into the platform. Seems like they are pushing updates to the web interface often, and really improving UX/UI.

gtaylor3• 11 years ago

Agreed on all counts. The only thing that is frustrating is that I haven't found a great way to keep up with changes aside from manually visiting their GCE release notes page.

bradgessler• 11 years ago

Convox is working on making it easier to spin up an ECS cluster. http://www.convox.com/

I've been playing with both and conceptually they're very similar. ECS is a little less sophisticated and extensible than Kubernetes, but that's by design since it only works with AWS services. For example, ECS uses ELB to abstract services whereas Kubernetes has its own service load balancer and service replication mechanism.

rce• 11 years ago

I'm a bit new to this, so not quite sure why you say Kuebernetes has its own load balancer. E.g. you can run Kubernetes on AWS. If you do then it uses ELB for load balancing.

devy• 11 years ago

So this Amazon EC2 Container Service Agent[1] is the AWS's own equivalent of Google Cloud Platform's Kubernetes? (Granted, Kubernetes is platform agnostic.)

[1] https://github.com/aws/amazon-ecs-agent

samkone• 11 years ago

This looks a lot like Mesos. But of course they will never admit that. Amazon and open source, a long history.

fidget• 11 years ago

It's more shared state than mesos, a strict 2 layer system, which is why the comparisons to kube are more common.

Tomdarkness2• 11 years ago

We've been using EC2 Container Service for a few months now and at first glance it appears great but we've ran into quite a few problems using it:

- You can only bind 1 port and 1 ELB to a service. This means, for example, if you have nginx listening on port 80 and 443 you need to manually configure the ELB and can't take advantage of automatically generated port numbers on the host (So you basically can only have 1 HTTP(s) server on a host)

- There is no way to cleanly decommission a host from a cluster. Say if you want to reboot or replace a server in the cluster you can't tell ECS to drain connections from the ELB and move the containers off the host

- Unable to specify rules for which hosts services should run on. For example you can't say that the service needs to make sure it has instances in X AZs or don't run multiple instances of the same service on the same host.

- No easy way to implement any kind of service discovery, you have to roll this yourself or set up loads of internal ELBs as a sort of service discovery.

- Worst of all the ecs-agent is very buggy. It has releases where it just keeps crashing [1] and leaving untracked running containers just hanging around to an issue where they released a new version with a newer docker library that changed how it handled certain parameters causing our entire cluster to fail until they hot fixed it with a change to how their API sent data to the client.

We're currently looking at alternatives for our docker infrastructure.

[1] https://github.com/aws/amazon-ecs-agent/issues/156

akanet• 11 years ago

I'm very interested in ECS's startup latency characteristics. Do you know offhand about how long it takes to provision a new container, and if this is subject to wide fluctuation?

Tomdarkness2• 11 years ago

I've not done any measurements in terms of latency but from my experience an agent on the host will ask the docker daemon to start a new container in the order of seconds after making an API request to ECS that calls for an additional container to be provisioned.

I'd recommend setting up a private registry within EC2 though, otherwise you'll have a fairly significant delay while it pulls the image from wherever and incur bandwidth charges for doing so. You'll still have some delay while it pulls the image from your registry running on EC2 but not as significant if it were externally hosted.

kordless• 11 years ago

Check out Giant Swarm. We have a hosted version you can kick the tires on.

osipov• 11 years ago

Check out the IBM Container Service: https://www.ng.bluemix.net/docs/containers/container_index.h...

darren0• 11 years ago

This is shameless self promotion but check out rancher.com. It's open source and allows you to spin up a container service like GKE or ECS but cloud independent.

ecliptik• 11 years ago

+1 for Rancher. As a in-house container management platform it's extremely simple to get setup and makes managing container stacks effortless.

Combined with Watchtower it's a potent container solution.

XorNot• 11 years ago

Wow - my company is getting into the EC2-Docker idea pretty heavily at the moment (and I'm going to wind up in charge of monitoring it). This is...not great to hear.

xfiler• 11 years ago

Check out also kontena.io .

ageek123• 11 years ago

This was already posted to HN about a month ago... https://news.ycombinator.com/item?id=9917339

Under the Hood of the Amazon EC2 Container Service

Related Stories