How Facebook uses Erlang for real-time chat

http://gregosuri.com/how-facebook-uses-erlang-for-real-time-chat

by gosuri • 16 years ago

37 20 16 years ago

melvinram• 16 years ago

I haven't used Erlang but I wonder if people will start blaming Erlang as "not stable" since Facebook chat is so unstable... the same way people said Ruby on Rails couldn't scale because Twitter was haven't problems scaling.

mononcqc3• 16 years ago

I hope not, because it's hard to blame only Erlang.

First of all, they rolled their own epoll-based server for the whole chat system.

Then, instead of using a 'sane' comet/long-poll implementation based on JSONP (to avoid limitations of same-domain requests), it seems that they used a scheme where the chat nodes have a fake subdomain like 'http://N.subdomain.facebook.com/.... I don't know how efficient this is supposed to be, but I haven't seen that too often.

Of course, it's wholly possible their Erlang implementation of the chat system is the reason why it fails. Personally, I have never had much trouble with their chat, although I haven't used it much either.

lukatmyshu2• 16 years ago

The reason that they use N.subdomain.facebook.com is so they can avoid the default "2 http requests per domain" rule in older browsers.

rjurney2• 16 years ago

Given how easy it is to scale an Erlang app to provide incredible reliability in concurrent systems - as that is what the entire language is optimized for - this is pretty unlikely.

axod3• 16 years ago

Is chat a 'concurrent system'? :/

gosuri• 16 years ago

I believe its the Frontend implementation of the Chat that's unstable. Meebo has done facebook chat implementation pretty well.

sjs382• 16 years ago

I thought the same thing. :)

warfangle• 16 years ago

Then again, the way Twitter overcame their scaling issues was to.. uh, well.. move off from Ruby for a portion of their stack.

aditya• 16 years ago

Er... why didn't they just use XMPP? I haven't used facebook chat much, but the times that I have used it, it has proven to be a lot more unreliable compared to gTalk.

mononcqc3• 16 years ago

Having evaluated some XMPP solutions (over BOSH), one of the problems I have had with it was about the openness of the protocol. You need to tweak the server a whole lot if you want to close down what your users can do and from where they can do it; it becomes complex to synchronize all the data from the backend of the site: who ignores who, privacy settings, friend lists, etc.

Another problem that I had encountered was about BOSH. From what I had read in the RFCs, the BOSH session is based on random seed that you hash over and over in SHA (say 200 times). The server then just has to SHA your message to find the hash of the previous message. This, coupled with the basic session identifiers effectively makes it relatively complex to have the chat working over multiple windows or tabs.

There might be other reasons, but those are the obvious ones to me.

suhail• 16 years ago

Isn't this a bit old and why does it go to posterous when it's taken from facebook.com?

tdmackey• 16 years ago

It's very old, and the posterous owner thinks its fine to blatantly copy entire articles by merely providing "From: http://www.facebook.com/note.php?note_id=14218138919 as the only indication that it wasn't original.

dylanz• 16 years ago

That was a fantastic read. This is indeed old news in terms of technology choices, but it was great to hear how they rolled the product out.

I almost didn't read this article due to the "isn't this a bit old" comments here, but, am very glad I did.

axod3• 16 years ago

FWIW, Their Facebook chat seems to be constantly wrong. Unless it's just my account. "50 users online", then you click on it, and it changes its mind and says 7 users.

aw3c2• 16 years ago

blogspam, actual URL is http://www.facebook.com/note.php?note_id=14218138919

How Facebook uses Erlang for real-time chat

Related Stories