A Node.js speed dilemma: AJAX or Socket.IO?

http://www.cubrid.org/blog/cubrid-appstools/nodejs-speed-dilemma-ajax-or-socket-io/#.UIpMM6a4eTc.hackernews

by kadishmal • 14 years ago

38 17 14 years ago

"transport" of Socket.IO plays major role (but, it's not smooth/plug 'n play). Benchmark missing note about chosen transport.

biscarch• 14 years ago

Was going to mention this as well.

socket.io falls back to long polling, so you have to be careful when trying to produce benchmarks based only on websockets. I didn't see this in the server code so I assume the default transports were used.

io.configure(ENV, function(){ io.set('transports', ['websocket']); });

more here: https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Soc...

Also, Firefox version would be nice to know.

CWIZO• 14 years ago

Why on earth would you use Socket.IO in a non persistant way?! That really doesn't make sense. But I'm glad you tested it out and didn't just assume that it was the fastest!

account_taken• 14 years ago

Aldipower• 14 years ago

Main advantage of Socket.IO is that you can do a push to the client over a persistent connection, that is not possible with Ajax in this way. Your results shows that, if you only need pulls from the clients, you could easily use ajax with all the landscape like jQuery and so on and it's not necessary to use Socket.IO.

luos• 14 years ago

"AJAX is usually faster than Socket.IO to “code”..."

I tried node.js and some modules a couple of weeks ago. It was pretty easy to set up and I had a "working" chat application in few hours with socket.io. It was 20-30 lines of code, provided callback to send data immediately to other clients.

I don't think polling the server every 500 ms is a good practice, say 5-10 sec or more can be reasonable, but with socket.io you got it instant.

olenhad• 14 years ago

You should have checked what protocol socket.io was using. socket.io fallbacks to xhr polling if WebSockets are not available in the client's browser, which is pretty much the same at ajax polling. WebSockets would probably give better results imo.

plasma• 14 years ago

I'm playing with socket.io right now, and having trouble benchmarking it past 1500 concurrent connections on an 8GB RAM VM with unlimited file handles, using https://github.com/michetti/socket.io-benchmark

Anyone have any experience / advice?

martzom• 14 years ago

You may find this thread useful: https://groups.google.com/forum/?fromgroups=#!topic/nodejs/0...

JimmaDaRustla• 14 years ago

I use SockJS - www.sockjs.org

lalc• 14 years ago

Yes, please. I switched from socket.io to SockJS a long time ago. I bought into it when I was first writing my app, but it is so unbelievably flaky and the design priorities are all wrong.

timc3• 14 years ago

Does that actually mean "No thank you"? I can't quite work out whether you have had a good experience or bad experience..

maarten• 14 years ago

Last time I checked Socket.IO doesn't support GZip compression. I found that, especially on mobile connections with larger datasets, it's just faster to use AJAX requests for client->server data requests, in combination with Socket.IO for fast server->client updates/notifications.

martindale• 14 years ago

It does in fact support gzip ( see: https://github.com/LearnBoost/socket.io/wiki/Configuring-Soc... ), but there's an outstanding issue with spawning the gzip process ( see: https://github.com/LearnBoost/socket.io/issues/932#issuecomm... ).

This should be fixed in the upcoming update to zlib.

kanaka• 14 years ago

Please post the full code and configuration you are using. Your results don't match what I have seen nor the expected results.

Some notes:

- Socket.IO supports several different transports, including AJAX. To have persistent connections you need to make sure that you are using WebSockets (native or Flash) as your transport. Are you certain that you are actually using WebSockets? Socket.IO attempts to automatically pick the best transport and unless you verify, the only difference you will see is lower performance.

- You need to compare Socket.IO using AJAX as a transport, vs raw AJAX so that you can control for how much overhead the Socket.IO library (client and server) is adding to the test. Actually, comparing all the Socket.IO transport options would be useful. Doing a Socket.IO disconnect is not a good way to trigger a non-persistent transport in Socket.IO; you are likely causing both client and server to do a bunch of extra cleanup (Socket.IO has some advanced features that require tracking connections/channels/etc).

- If Socket.IO is using WebSockets as your transport, then your "Socket.IO non-persistent" test is still testing WebSockets, not AJAX or one of the other real non-persistent connections. I would not be surprised that WebSocket connection setup and teardown might be slightly slower than AJAX connection setup. HTTP connection setup has been around since the beginning and had a lot of time to be optimized. WebSockets are relatively new and connection setup time is not something that people will probably ever care about (WebSockets are not designed to be used this way).

- You should really use something other that Date() for your timers. The accuracy of Date is close to the latency of WebSockets themselves and so I strongly suspect that if you are doing this to localhost, what you are actually measuring in "Socket.IO persistent" vs "AJAX" is the Date rather than the connection latency Note that the latency per request in either case stays around 3ms and the Date object returns a value in milliseconds. The "Socket.IO non-persistent" is probably just a measure of Date with the overhead of Socket.IO cleanup. I suggest using high performance timers: http://updates.html5rocks.com/...

- You need to do your test between two different computers preferably with at least a switch in between. There are a lot of network stack shortcuts that happen on modern computers when you communicate over the localhost/loopback network.

- You are waiting 500ms between requests. I would suggest making that a variable. Interesting things will start to happen as you lower the interval. Also, as part of that, I think you probably want to put your start counter into the message data and then extract it on the other end to do the calculation. This is important because as you lower the amount of time you may start getting overlapping sends and receives (which is where things start getting interesting).

- You should really generate your random string outside the send block so that you aren't adding latency and CPU usage to the performance loop. Also, it would be useful to be able to tune the size of the data being sent to both larger and smaller sizes than 4K.

Finally, you mention in your lead off that one of the problems with WebSockets is that they are persistent so you may reach your limit of concurrent connections compared to AJAX. But in actual practice, with well designed and configured servers, you will reach server limits with AJAX requests FAR sooner than with WebSockets.

For example, consider an application that occasionally sends updates to a few clients and has 10,000 clients "connected".

AJAX is the worst case because every client has to establish a new connection to poll the server for data so to get sub-second latency for updates, the clients would need to make a new connection every 500ms or so. This means the server is having to setup 20,000 new HTTP connections every second.

Using long-poll (server hangs onto the client connection if there is no data) the client will probably have to re-establish a connection every 30 seconds (depending on server/firewall config/etc) to poll the server. This is much better, but that is still over 300 connections being established every second to the server (and only if they are evenly spread out). Also it's worth noting that with a long-poll scenario, you have most of the clients with established connections most of the time. So you'll run into connection limits just as quickly (actually sooner for various reasons) than with persistent connections.

With WebSocket connections, there is the initial connection setup to connect the clients, but once the clients are connected, if the server has no data to send, there is essentially NO traffic between client and server.

Looking forward to seeing your code and configuration. How about posting to github so that others can try and replicate your results and suggest improvements?

jugglinmike• 14 years ago

The OP doesn't mention expected client count, but the number of concurrent connections is a crucial factor that these tests overlook.

I'm currently doing some Socket.io stress testing, and it's clear that XHR-polling is significantly harder on the CPU for the reasons you've mentioned. This detail is lost in the noise for single-client tests, but it will become increasingly relevant as more clients connect.

js4all• 14 years ago

I am sceptical. Other socket.io vs. ajax tests show opposite results.

https://github.com/monteslu/jsonrpctest

Here socket.io is at least 2-3 times faster than ajax.

A Node.js speed dilemma: AJAX or Socket.IO?

Related Stories