Node.js fanout pubsub server for real-time apps

http://github.com/jazzychad/fanout.node.js

by jazzychad • 16 years ago

54 7 16 years ago

I'm curious about the "fanout" name which immediately made me think of a message broker like RabbitMQ on the backend somewhere. I guess "fanout" here means 1 client to fanout.js to N >> 1 other clients.

There are a number of similar projects popping up like this one as of late for Node.js. NodeRed, fanout, dealer, etc.

That's great. Node.js makes previously "complex" things a little easier to setup and play with.

brainproxy2• 16 years ago

I'm wondering if the author or other readers have looked at this node.js library:

http://github.com/jcoglan/faye

Coglan has put together code for ruby and eventmachine too, if you prefer those tools over javascript and node.js.

Use faye in combination with redis-node-client and you've got an efficient means to create a high-volume, low latency work queue.

40+ mbps of http traffic and upwards of 20,000 redis requests per second can be serviced by a couple of node processes, using only a few hundred MBs of RAM and a plain-jane Linux VM with access to multiple cores. Tests are ongoing....

brainproxy2• 16 years ago

I mention that library specifically as it's a pubsub implementation for http/javascript too.

stefs• 16 years ago

can anyone explain why the message format uses fancy control chars like

    <channel_name>!<message><newline_character>
    foo!hello, world

why not json?

    {cname: "foo", msg: "hello, world"}

i mean, parsing fancy format may be a bit faster and needs less library code; and, of course, overhead. but from personal experience i know that fancy control chars mostly lead to perlish charsoup as soon as new features are implemented.

    foo!hello, world##from_id:213/5|sendername:chad//validuntil{20100429105700}?debug=true&nlformat=CRLF

i guess, overhead; am i right?

jazzychad2• 16 years ago

I don't consider ! to be a fancy control character. It's just simple.

Outputing/requiring json adds extra complexity and restriction that isn't necessary. Although, nothing says you can't send the messages as json (which is, in fact, what I do in my applications!). Letting the message format be completely free-form means you can treat it however you like.

The idea was to make it incredibly simple, and abstract enough (meaning no extra/unneeded logic baked in) so that it could be used in the widest set of applications. The application you build with it may well use json as its message format, but I saw no need to require a certain format.

Here is an example of a message I'm using in the 4squrevision example:

    4sq_checkins!{"iid":17295,"tid":13057779551,"location":"Maths Journey","sqlink":"http://4sq.com/cGG46G","user":"greeenpea","lat":null,"tstamp":"Thu Apr 29 08:20:33 +0000 2010","lng":null}

Simple.

adrinavarro• 16 years ago

Where's the difference between using this pubsub implementation vs. using Redis one? I think that overall Redis one is way better (stability, performance).

jazzychad2• 16 years ago

I don't know. It may be better, I haven't looked at it yet. I wrote this implementation last December when redis didn't have pubsub yet. I thought it would be nice to share it as an example of the simplicity and power of node/javascript.

How can you state that the stability and performance are better? I've been running one instance of this node pubsub server for 5 months without restarting it. You may be right, but have you actually benchmarked it?

Node.js fanout pubsub server for real-time apps

Related Stories