Speed Limit of PaaS - 64K TCP Ports

http://blog.mudynamics.com/2011/08/26/speed-limit-of-paas-64k-tcp-ports/

by kowsik • 15 years ago

20 33 15 years ago

blog.mudynamics.com

There are several assumptions in this article. The biggest is that, at scale, the same NIC will be used for both front-end and back-end stuff. What if it's not? Then you aren't penalized for your back-end ports on the front-end because they're on different routes.

Also, why is Node special here? Did people not notice Comet, Athena, Flash, Java, etc.? We've been doing long-polling and extended socket usage for a long time, not to mention browser connection reuse. This port count problem hasn't been a problem at all yet.

Finally, scaling up to 60k useful active concurrent connections on a single box is pretty damn tricky. Usually, you'll be splitting that kind of load, and doing NAT, and all of a sudden you no longer have to worry about port exhaustion.

kowsik9• 15 years ago

My observation/assumption is that your app is running in a multi-tenant PaaS environment, where mysteriously you start getting bind errors because the ephemeral port space is shared with other tenants.

MostAwesomeDude4• 15 years ago

You have observed this? There's a trick to the way sockets work: They track a bunch of data about each connection, not just the local port number and IP. You're talking about exhausting a space of 60k+ connections/user, not connections/server.

If you've actually seen this in the wild, then I'd like to know what you're running, what you're selling, how you're doing so remarkably well, etc. This sounds like the best problem to have. :3

superjared4• 15 years ago

This is bollocks. The argument is that you can only have ~60k connections due to the limit of ephemeral ports. This applies on a per-client basis, so you can have nearly limitless connections. I wrote about this here under "The 64k connection myth":

http://urbanairship.com/blog/2010/09/29/linux-kernel-tuning-...

kowsik9• 15 years ago

Read the blog again. This is bollocks if you are running your own server where you can be multi-homed, add loopback aliases etc and crank up the memory to support 100,000+ sockets. I'm talking PaaS where your app is an multi-tenant environment with no root access.

superjared4• 15 years ago

You don't need loopback aliases for > 64k sockets. You can have 64k connections to your server per ip address that connects.

EDIT: Formatting

moe• 15 years ago

It's an esoteric limit either way.

In just about any real-world scenario (including PaaS) you'll be hitting a whole range of other bottlenecks long before this one.

ericflo2• 15 years ago

I wasn't able to understand what the author was trying to say under "Connection Pools"--is he suggesting that the database should be written using node.js so that, presumably, it can "scale out" and not require a connection pool?

superjared4• 15 years ago

Either that, or allow for >64k simultaneous connections from node so that each of his requests has its own connection.

The author doesn't seem to understand that more connections to a database does not equal higher throughput.

kowsik9• 15 years ago

We had a node.js app where each inbound request would go through a set of HTTP connections to the back-end couchDB. When the concurrency reached around 1,000 then there were 1,000 requests made through 10 couchDB pipelined connections. The result was the web requests were starting to get slower and slower. Think of a 6-lane road merging into a 1-lane road. Congestion.

sapphirecat2• 15 years ago

The "speed limit" is on the outbound connections from Node to the service. Node calls connect(), and receives a new port exclusively for that one connection.

A server facing the Internet can serve lots of clients because those clients have plenty of IP+port combinations to go around on their end, to allow the server to tell the difference among them even though it only has the one IP+port on its end. But 60K node.js connections from one machine, the frontend server with a single IP address, to a single IP+port on the backend server, do not have that luxury. All that identifies the connection now is the port number on the Node server, so it must be unique per connection.

Connection pools attempt to mitigate the problem by inserting a manager (the pool) in the middle, to accept larger numbers of requests from Node and try to schedule them on a lower, sustainable number of connections to the backend. At least on an RDBMS, transactions require the app to have exclusive use of the connection for its request, so when all the real connections are scheduled out, new requests have to wait for an old connection to be relinquished.

EDIT: Going back to the blog post, it said, "You really need your back-end services to scale out with node.js." Which I think means, your back end service should have multiple IP addresses, to alleviate the bottleneck described above.

kowsik9• 15 years ago

I've added a picture to clarify which ports I'm talking about. Hopefully this clarifies things a bit.

prodigal_erik• 15 years ago

Strange to warn of a problem having already dismissed the solution (pooling). Unless your backend actually has fifty thousand cores, making fifty thousand concurrent connections to it accomplishes nothing more than moving the bottleneck around while defeating TCP flow control. Capacity is always finite, some part of the system is inevitably going to play the role of a bounded queue, and you want that visible and close to (preferably in) the frontend so you can shed load through degraded service.

DasIch• 15 years ago

Before you even come close to that limit any decent PaaS solution should move your application to a less-used server or spawn your application on several servers by load alone.

(Assuming you invested enough money.)

kowsik9• 15 years ago

Talking about port exhaustion is, I realize, a little confusing because there are the web requests coming in and the web-tier is making outbound requests to other services. Adding an image to the blog to hopefully clarify things.

salem2• 15 years ago

Also, if there is high churn on those outgoing connections, then the problem is a little worse. The connections per second will be much lower than 60k, because of the timeout before the port number is reused for a new connection. Although, this timeout can be tuned.

Speed Limit of PaaS - 64K TCP Ports

Related Stories