Plurk Comet: Handling of 100.000+ open connections

http://amix.dk/blog/viewEntry/19456

by niels • 17 years ago

44 38 17 years ago

Hmm, was twisted not the next big thing that everyone was rewriting their app in? What does he mean, it does not scale?

mahmud• 17 years ago

Not a python programmer here and I'm only guessing after a cursory look recently. Twisted developers are layering a higher-level "architecture" over the basic system calls they have wrapped. It's just a few calls down there (serial and scatter-gather I/O, multiplexing, etc.) but twisted adds higher level data structures, and design-patterny conceptual models to shield the pythonista from Unix.

Some of that gotta add some fat; which might be OK for the great majority of the people, but the few who need raw performance might end up cutting some bacon off.

That dude at amix.dk actually knows his stuff.

gertburger• 17 years ago

Being a python programmer I am a bit worried that one (In this case) needs to use java to overcome a scaling problem. From my experiences with twisted, which is on much smaller scale than the article, I have never found twisted to be anykind of bottleneck.

It may be that since twisted(python) can only use one cpu effectively per interpreter(GIT et al) that it got left behind java which can easily use multiple CPUs for threads.

A slightly different architecture might be required then where multiple python processes are used.

amix• 17 years ago

So far at Plurk we have struggled to overcome scalability issues - - and none of them are due to Python, even thought we use Python a lot.

Most of the problems we have seen have been related to the database. It was and still is the biggest bottleneck. And all the people I know that drive big sites will confirm this.

This all said, one should evaluate problems and not blindly use Python for everything. Python is a great language, but some other language is a better choice for some problems. And here Java and specially Java's NIO library is a much better choice for doing a server that should handle tons of open connections.

majek• 17 years ago

You don't have to use Java. That's exactly the reason I hacked together EvServer: http://code.google.com/p/evserver/wiki/Documentation

Because of GIL, there is no point of using multiple threads in Python. Scaling must be done through adding more processes. Than an IPC communication starts to be an issue - that's what messaging middleware is for. The sooner you integrate messaging with your project - the better.

Once you have a messaging platform - comet can be done in any technology you prefer, it really doesn't matter. That's because the scaling complexity is handled by the message broker.

d0mine• 17 years ago

s/GIT et al/GIL et al/

henryl• 17 years ago

Has anyone had any success with orbited?

axod7• 17 years ago

afaik orbited is written using twisted, in python.

sho3• 17 years ago

That Omegle site uses orbited, seems to work OK with reported load of a few thousand connections.

jasonkester5• 17 years ago

This is a nice writeup that gives some good insight into what you're up against if you want to implement Comet today. It's a cool idea, but there's really nothing out there that you can use to do it off the shelf. It's great that he's gone through and actually tested out a bunch of these candidate technologies under load, so the next guy will at least know what not to try.

I have faith that Comet will have its day eventually, but clearly it's not quite time. With the current crop of production quality web servers, handling 100k requests per second (polling) is a solved problem. Keeping 100k simultaneous connections alive (comet) is by no means solved. It'll be cool when it happens though.

axod7• 17 years ago

>> "I have faith that Comet will have its day eventually, but clearly it's not quite time."

??? Many many big sites are already using comet, and have been for some years, so I'm not sure I understand your comments.

jasonkester5• 17 years ago

Correct. If you're Google, and can devote a team to build and maintain a webserver specifically designed to handle Comet, then you can use Comet.

If you're small and need to use off-the-shelf tech to get your thing up and running quickly, you're still best off going with Polling today.

A quick example would be to look at Thinkature vs. Twiddla. They had a team of two, one of which spent an entire year building a web server from the ground up so that they could use Comet. Twiddla only has me as a developer, and I'd rather spend my time improving the product, so we use Polling.

One year later, Twiddla has a ~300ms lag between when you draw a line and when it appears on a remote screen, whereas Thinkature is out of business. I don't think it's that black and white of a tradeoff, but hopefully you see the thrust of what I'm saying.

One day soon, there will be a mod_comet for Apache, or IIS 9 with Comet support built in. That will be the day it makes sense for small, lean teams to build a business around it.

est• 17 years ago

http://www.alexa.com/siteinfo/douban.com

AFAIK douban.com seems to be the largest (pure) Python site.

rams• 17 years ago

Douban uses a lightweight python framework called Quixote. http://www.mems-exchange.org/software/quixote/

Here's the mail sent about two years back to the quixote users list: http://mail.mems-exchange.org/durusmail/quixote-users/5657/

tezza• 17 years ago

SPEED :: In defence of Java, it is very quick when done right. See Kilim[1] for examples.

SUPPORT :: On top of the speed Java has a supported hardware stack (Solaris). So if doing 100K connections is important to you, you can find engineers who can diagnose difficult problems on your whole stack.

--------------

[1] http://www.youtube.com/watch?v=37NaHRE0Sqw at 23:23 . Java is quick without Kilim too, this is just an example with stats.

ars• 17 years ago

For the non UK people 100.000 = 100,000 or 100 000 if you want the ISO version.

simonw• 17 years ago

We use 100,000 in the UK too - 100.000 is used by some European countries (Spain is one, not sure about the others).

judofyr• 17 years ago

FYI, in Norway we use 100,000 for 100 and 100 000 for 100000 (no dots).

Raphael• 17 years ago

No, the poster was just being very precise.

axod7• 17 years ago

Number of open connections is a little bit meaningless until they're actually under load, sending and receiving packets in a real world situation.

I'd agree, Java+NIO is a great system. Extremely fast+scalable.

davidw4• 17 years ago

Not knowing much about Java's NIO, how do you handle 'long running calculations'? One of the great things about Erlang is that you can sit there doing some big huge long loop, and it won't block the rest of the system because it's got an internal scheduler. One way to do that kind of thing with C or Java is simply to manually divide your big long calculation into smaller chunks that yield control at regular, brief intervals. Does Java provide anything higher level?

axod7• 17 years ago

What sort of long calculations? If you're doing massive number crunching, you could just have a queue->thread->callback for it if it's not easy to break up into bite size chunks...

I don't know of anything else built in.

sho3• 17 years ago

Well, it's not that meaningless. Comet is basically long polling; those connections aren't really doing much of anything until something turns up on the back end to be pushed down them, at which point the connection is usually closed and then re-requested by the client. Maybe with the occasional ping, I've seen some people do that too.

How much memory and CPU is required to simply hold open the connection and sit there is definitely of interest to people. Of course, I'd like to see real world tests too, but that might be rather difficult to pull off without specially written, extremely efficient code on the test side.

axod7• 17 years ago

Sure. It's the "basic" level requirement. Holding open a tcp connection should be pretty cheap though unless something is badly wrong.

sho3• 17 years ago

Interesting. I'm writing a comet server right now, using Ruby's EventMachine. I might do some tests to see how many connections I can stretch it out to; unfortunately I have the feeling that local connections are not going to quite match up to real live outgoing ones.

If I was doing it For Realz™ I reckon Erlang would be the go, still haven't gotten up to anywhere near that level of proficiency though. 37Signals recently rewrote their push server in Erlang and reported great results.

davidw4• 17 years ago

This article seems relevant, BTW:

http://news.ycombinator.com/item?id=334506

iamelgringo• 17 years ago

There was a nice write up on Erlang based Comet apps here: http://cometdaily.com/2007/12/14/getting-started-with-comet-...

Has anybody tried comet using Yaws?

c00p3r• 17 years ago

Is an actual c++ implementation of the java's nio is strictly better than nginx's one? Is it using epool() and sendfine()? Isn't all those java's abstraction leyers are overhead?

It seems like yet another attempt to put some more air into java's bubble.

Plurk Comet: Handling of 100.000+ open connections

Related Stories