Fast polling using C, memached, nginx and libevent

http://amix.dk/blog/viewEntry/19414

by amix • 17 years ago

71 18 17 years ago

liuliu• 17 years ago

If it is a poll server, why not write a lite script of lua inside nginx or just a plugin of nginx? That would take full features of nginx and I guess the result would be better.

staunch• 17 years ago

I think modifying the nginx memcached module would have been ideal. Running a new server that does almost nothing does not seem like a great way to go.

davidw2• 17 years ago

It'd also be interesting to see the whole thing done in Erlang, just for fun.

davidw2• 17 years ago

... or apparently not, for those with little intellectual curiosity about a task quite well suited to Erlang.

bcx• 17 years ago

What is the advantage of polling over comet? I guess the real question is how often is a user re-polling the same value, without it changing.

-Ben

lhorie3• 17 years ago

Polling works out of the box with pretty much all web servers - the downside is that it's significantly slower and more resource intensive than comet.

simplegeek• 17 years ago

>> the downside is that it's significantly slower and more resource intensive than comet.

Any data or story to back that argument? Just curious.

axod5• 17 years ago

>> "From what I have seen, all comet solutions are very hacky, quirky and very hard to scale. The scaling is especially hard since the whole web-platform is built around the request-response model."

I disagree about this. There's nothing hacky/quirky about comet, and it can scale really well. Some numbers from a Mibbit backend:

  Peak connected users: 4,000
  Peak HTTP Req/s: 1,500

  HTTP requests per day: 44 million
  Packets sent per day: 42 million
  Packets recv per day: 51 million

  (On a single 1.4GB VPS). Using *OMG* java!

If you're not using comet, that HTTP req figure would be way higher, as a far larger number of HTTP requests would be unuseful (no data available). Wasting time, bandwidth, etc As well as obviously adding latency. With comet, the data is available in most instances as soon as the server has it ready. With standard polling, the user has to wait for the next poll.

Comet is pretty easy to scale once you understand exactly what is going on, and how best to organize things. Mibbit uses a keep-alive long polling XHR setup which works really well. The keep-alive ensures that most of the time, you just have a tcp/ip connection open from server to client, with some HTTP request/response spam every so often, and the actual packets.

Having said that, I wouldn't use any of the open source comet things I've seen out there.

(I do plan to open source Mibbit at some point, which might be useful for people).

lhorie3• 17 years ago

Out of curiosity, what transport are you using? Long polling or long-lived iframes? I find that the former doesn't give me low enough latency for my purposes and the latter tends to get quirky for me when I have a high spiked data throughput.

axod5• 17 years ago

long polling XHR. Latency is extremely good if the server software is up to the task. As soon as the server sends an HTTP header and data response, it's there on the XHR response. (Also iframes mean memory leak, and possibly 'loading' spinner. Neither of which you get with XHR).

You can also do some neat optimizations like decide that a latency of 100ms is ok enough, and so delay sending any data for up to 100ms to see if you can piggy back on the next recv request and reduce the HTTP request count by 1. Also in order to batch up sends. For example:

  Without optimization--
  0ms Long polling Recv
  800ms Send some data
  900ms Long polling Recv (After some data was received)

  With optimization--
  0ms Long polling Recv
      (Send queued for up to 100ms)
  900ms Data sent, in Long polling Recv request.

blasdel• 17 years ago

What connection scheduling mechanism are you using? Is it non-blocking asynchronous, or are you using threads? What's the resource overhead per open connection?

axod5• 17 years ago

non blocking async. Single thread. There's a bit of memory used for buffers, but nothing major. It's been up to around 14k connections total so far in the real world, more in testing.

amix• 17 years ago

I think Java is a pretty good choice, especially for doing comet or any other problem where performance is required. Today Java's perfomance almost matches the speed of C or C++... I have some time ago played around with using Jetty's continuations and it's pretty impressive (and highly scaleable).

In my next iteration I'll probably try to go with the long polling model and I'll probably explore C and Java.

Thanks for sharing your architectural notes.

axod5• 17 years ago

Cool :)

Forgot to also mention CPU usage is negligable. Memory can be a challenge with Java, but often solved by reducing object churn, tidying up better etc. The java.nio package really is pretty awesome.

chiffonade• 17 years ago

That architecture diagram makes no sense.

Fast polling using C, memached, nginx and libevent

Related Stories