Websockets 101

http://lucumr.pocoo.org/2012/9/24/websockets-101/

by the_mitsuhiko • 14 years ago

289 50 14 years ago

lucumr.pocoo.org

romaniv3• 14 years ago

Why do standards like WebSockets don't include anything that can be used in pure HTML? This would be a great tool for building efficient dynamic apps, if you could "submit" forms without going to a new page, and then get incremental page updates as a response.

j2labs• 14 years ago

Because it's a sockets layer. You could very easily send HTML over it if you want.

dbaupp• 14 years ago

What is so bad about a little Javascript?

And, how does one get sufficiently customisable incremental updates without having to standardise (and implement) a huge complex new language to control them?

romaniv3• 14 years ago

What is so bad about a little Javascript?

What's so bad about having a declarative way of doing something that a lot of people will find useful?

And, how does one get sufficiently customisable incremental updates without having to standardise (and implement) a huge complex new language to control them?

By using element IDs. Every element in the incoming frame that has and ID that matches and ID of an existing element replaces it in the document. People have been doing this for years. It's called AHAH (Asychronous HTML and HTTP). Doing this without any custom code and with every update being as cheap as sending WebSockets frame would be absolutely awesome. It could be used to create very sophisticated and dynamic applications with very little complexity either on the client or on the server side.

xionon• 14 years ago

I'm not sure I see the use-case there as being more useful than JS based WebSockets. Writing a general JS library to do what you suggest would be pretty straightforward, and the server-side work wouldn't be any different (serving partial HTML based on request type).

If your users are worried about security / performance enough to turn off JavaScript, they probably wouldn't want "HTMLSockets" enabled, either.

If your users are browsing with a browser so old it doesn't have a reasonable JavaScript engine, then it wouldn't support "HTMLSockets," either.

Basically, there's nothing you could do with "HTMLSockets" that you can't do with WebSockets, and with only a small layer to make it so. However, there's a whole host of utility that you get from WebSockets that you could never get from HTMLSockets, or would require a huge amount of hacking in order to get it to work.

And all this is beside the "pureness" argument, where one might say that HTML is for data, CSS is for design, JavaScript is for behavior; WebSockets are behavior, not data.

romaniv3• 14 years ago

This is like asking why we should bother having any (new?) HTML or CSS standards at all, when you could write a custom rendering engine in JavaScript for every website.

There is tremendous value in having a solid, usable foundation for web development that doesn't rely on an imperative language to work. To start with, declarative features are semantic and allow for semantic upgrades on wide-scale basis. Do you like your Firefox/Chrome spell-checking in text areas? Do you like them being resizable? Well, this wouldn't be possible if all textareas were some JavaScript hack. (Hey, with only small layer you could actually fake a textarea. Does that seem like a good idea in retrospect, though?)

Also, it would be great to be able to develop simple dynamic web applications (or prototypes) without doing something complex. Progressive enhancement is buried far too early.

If your users are worried about security / performance enough to turn off JavaScript, they probably wouldn't want "HTMLSockets" enabled, either.

I frequently browse with JavaScript blocked by default, and I can tell you right here that this assumption is invalid. There is a world of difference between JavaScript library and a standard, declarative technology. The latter is more likely to be faster, have less bugs, no side-effects and be secure.

TazeTSchnitzel3• 14 years ago

This exists in Firefox. HTTP Server Push. https://en.wikipedia.org/wiki/Push_technology#HTTP_server_pu...

btmorex• 14 years ago

If anyone finds themselves needing to write a Websockets implementation, there's an awesome protocol test suite at http://autobahn.ws/testsuite

I wrote a C++ implementation for a side project and with the aforementioned test suite I actually found it pretty easy to get to 100% compliance. There's some ugliness in the protocol because of proxies, but it's definitely not the worst protocol in the world. The only big missing feature is compression and there's a proposal for that (you could certainly do application level compression, but I'd rather avoid writing compression code in JS).

zafriedman• 14 years ago

An even quicker introduction to the final specification of WebSockets: http://socket.io.

P.S. I'm aware that a) this doesn't help anyone not using Node.js on their server (it's not even part of my production stack at work (yet!), even though I'm bringing it to light here) and b) it's more than just WebSockets, for instance it will gracefully degrade on legacy browsers.

I just can't miss an opportunity to sing its praises because it has so many benefits over the simple implementation.

chrisduesing• 14 years ago

It is not actually limited to Node. https://github.com/learnboost/socket.io/wiki

Though some of these libs do tend to lag in version support.

peterwwillis4• 14 years ago

Encryption is not a panacea. As more websites use HTTPS, more corporate networks are installing content filters that passively filter HTTPS content.

Yes. I just said corporate networks are spying on HTTPS connections.

It's really simple to implement. Get some kind of web proxy that has support (Websense is pretty popular, amazingly) and generate a root cert on the box. Then use a group policy on your AD server to distribute the root cert to all the client systems. Now Websense can decode the HTTPS traffic by issuing its own fake certs to clients and handle the "real" HTTPS handshake on the frontend proxy side.

Result? Your websockets are still going to get fucked with by proxies. Can we please stop building fake protocols on top of real protocols now?

the_mitsuhiko3• 14 years ago

This won't work with our games. We verify that the certificate was created by our CA and reject anything else. Someone would have to patch the game to bypass that. It also won't work on mobile phones that people use themselves.

peterwwillis4• 14 years ago

How? Is this some JavaScript magic i'm unaware of? Because the actual connection from the proxy to your site uses your site's valid certificates; only the client to the proxy uses a "self-signed" one. Are you using client certificates?

kibwen2• 14 years ago

Interesting, I actually hadn't considered the possibility of an employer issuing fake certificates to peek at its employees' encrypted sessions.

For anyone unnerved by this, you can still get around it by routing your traffic via SOCKS proxy to an external host, but of course this assumes that 1) your employer isn't blocking SSH, 2) you have the authority to install PuTTY and Firefox (not sure if Chrome has SOCKS proxy support) and 3) you have an external server to proxy through (if you have a Linux machine somewhere, you're probably already configured to allow this). And make sure you route your DNS through SOCKS as well.

0xbadcafebee• 14 years ago

If they're proxying HTTP traffic usually they'll block SSH and other protocols. To work around this use an HTTPS/SSH multiplexer[1][2][3] or a simple reverse proxy to accept HTTP[S] connections and connect them to an SSH server on the backend. For the ssh client use a ProxyCommand tunneling app[4][5][6] to turn an HTTP[S] proxy request into a two-way socket, and you have an SSH connection over HTTP proxies.

[1] http://www.rutschle.net/tech/sslh.shtml [2] http://www.pond-weed.com/multiplex/ [3] https://github.com/stealth/sshttp [4] http://zwitterion.org/software/ssh-https-tunnel/ssh-https-tu... [5] http://proxytunnel.sourceforge.net/ [6] http://www.agroman.net/corkscrew/

jeremya• 14 years ago

Unless I'm mistaken, a cursory examination of the certificate supplied to your browser should show that it is one of those "mitm" root certs and not the certificate for the website you are browsing. That should make it simple to at least know your encrypted traffic is being hijacked.

peterwwillis4• 14 years ago

A cursory examination of the certificate will appear to be exactly the same as the "real" one, with the exception of lengthy obscure keys and numbers that nobody would identify unless they compared them to the original side by side.

The proxy vendor MAY provide its own issuer description in lieu of faking the original, but it by no means has to. Fake certs can look just as authentic as real ones if your browser trusts the root cert that signed them.

If you want to verify a certificate is signed by a "real" trusted CA, use a program that traces the chain of trust against a list of CAs that ship with browsers. I have such a tool here: https://github.com/psypete/public-bin/blob/public-bin/src/ne...

tlrobinson• 14 years ago

I noticed a company I worked at had installed their own root cert on the machine they gave me. I disabled it, but didn't get any warnings about invalid certificates, so I assume they weren't using it to snoop HTTPS traffic.

Unless there's a way to detect whether a cert is installed somehow?

rprasad• 14 years ago

What you are proposing is generally illegal in the U.S. Companies are allowed to snoop on non-encrypted web traffic, because there is no expectation of privacy, but different standards and rules apply to encrypted sites, i.e., including banks.

Also, since bank websites and email are the most frequently accessed encrypted sites, a company would be exposing itself to ruinous liability if a data breach led to the release of private employee information, i.e., bank records. No company would take on that sort of liability, and you can bet that any competent legal counsel would make the company aware of this before they implemented something like this.

jschrf• 14 years ago

Excellent article, thank you for sharing.

As someone who has implemented a WS server and client and would like to be able to host them in Amazon's cloud, the notes about ELB are worrysome.

gabrtv• 14 years ago

Websockets through ELB works fine if you use the TCP listener. You lose out on some HTTP-specific features like Cookie-based session stickiness, but it works great otherwise. We've been using Websockets/ELB in production for almost a year now.

tinco2• 14 years ago

Very nice readable article. Having dabbled with implementation a bit I agree with most of his points.

The author complains about websockets being frame-based instead of stream-based, because he argues if he would want a frame-based system he could easily model it on top of it.

I think frames are awesome. It is easier to build a stream on top of frames than it is to build frames on top of a stream in my opinion (you don't need an extra protocol for starters). Besides that almost anything you would like to do can be expressed in frames, saving most of us a lot of headaches.

I would like to add that websockets are not the end-all answer for web-game programmers. It makes some stuff easier, and reduces some overhead, but it's still on TCP, meaning realtime multiplayer games are still as good as impossible to implement.

Sadly all the issues with proxies go for UDP and then some, so I don't think we'll see any ubiquitous browser UDP protocol anytime soon :(

the_mitsuhiko3• 14 years ago

> I think frames are awesome. It is easier to build a stream on top of frames than it is to build frames on top of a stream in my opinion (you don't need an extra protocol for starters). Besides that almost anything you would like to do can be expressed in frames, saving most of us a lot of headaches.

Maybe I'm just old fashioned but I really don't like the idea of frames forced onto me. I much rather get my data in small pieces and can already start processing data as it comes in than having something buffer up everything and then giving it me as a large chunk. Streaming json for instance over websockets is a lot harder than TCP directly.

ishbits• 14 years ago

IMO the built in framing is one of the things I really like about websockets. I'm sure it will become my transport of choice for non performance critical messaging even when a browser is not one of the peers.

wmf• 14 years ago

WebRTC is already there if you want UDP.

christkv2• 14 years ago

well does it though ? Data channels seems to use SCTP over DTLS over UDP which would create a sudo "TCP" layer on top of the UDP layer as I understood it :(.

Negitivefrags• 14 years ago

My issue with it is that TCP is already a stream based protocol on top of a frame based protocol.

This is a frame based protocol on a stream based protocol on a frame based protocol. Now if you want a stream based protocol you would need to add another layer on top of that. Madness.

TazeTSchnitzel3• 14 years ago

(Shameless plug...)

I have a list of websocket libraries and a small echo server demo here: http://ajf.me/websocket/#libs

kibwen2• 14 years ago

That first link there to your Github repo is broken: https://github.com/TazeTSchnitzel/WebSocket.us

TazeTSchnitzel3• 14 years ago

Oops. I moved it to my main website's repo: https://github.com/TazeTSchnitzel/ajf.me/tree/master/htdocs/...

bowmessage• 14 years ago

Loving the article! Just thought I'd point out: "needles complexity"

rizwan• 14 years ago

For folks thinking about using Socket.IO for websockets, it works very well, but remember that most open-source implementations ONLY implement WebSockets and NONE of the other fall-backs that are in the Socket.IO spec. So without some work, these libraries are often useless for mobile development, for instance.

digisth• 14 years ago

I'm using SockJS (client and server: https://github.com/sockjs/sockjs-client) for project I'm working on, which does implement a variety of fallback methods (iframe tricks, JSON polling, etc.) and it's been working pretty well (some work better than others.)

For those who to opt for these libs, I recommend the following for protocols/fallbacks:

protocols_whitelist: ['websocket', 'xhr-streaming', 'iframe-eventsource', 'iframe-htmlfile', 'xhr-polling', 'iframe-xhr-polling']

This gets you a working system in pretty much every modern browser supporting WS, and /usually/ works pretty well in IE8/9. Just try to avoid page refreshes in those two browsers (very spotty results if you don't - I'd recommend putting up a message for your users) and implement the workarounds for finicky browsers (things like catching ESC in Firefox.) I've been pretty happy with it so far.

kaiserama• 14 years ago

Pop-up Video Excerpt:

The author (Armin Ronacher) also authored Flask (Python web micro framework) and its template engine Jinja2. I've used Flask for a number of my most recent projects and really have enjoyed working with it.

Back to your scheduled broadcast!

chadrs• 14 years ago

Man, I would have loved to have this about a year ago. Although then I would have missed out on some of the hilarious drama on the websocket mailing list (e.g. http://www.ietf.org/mail-archive/web/hybi/current/msg08668.h...)

Evbn2• 14 years ago

> Messages sent from the client to the server are obfuscated with a basic transmission mask of 4 bytes which is sent within each websocket package.

What is this? Some pretend security feature?

pjscott• 14 years ago

The purpose of the masking is to prevent malicious applications from being able to send arbitrary bytes over the wire, for fear that they may be able to screw with buggy proxies.

_urga3• 14 years ago

Browser vendors must start to move away from top-down innovation, where they hoard APIs such as TCP and UDP and release sub-standard specs in their stead. Instead they must expose only the bare minimum OS low-level APIs (TCP, UDP, POSIX), keeping the surface area as small and powerful and direct as possible, and then let open source grow around this. Innovation needs to be decentralized, bottom-up. Not designed by committee.

For that to happen, we need to stop conflating "web apps" (trusted, installed, annointed with machine power) with "web pages" (accessed by single link click). At present, Web Apps are suffering and crippled by being lumped under the same security policy as web pages. But Web Apps need to have access to raw machine resources in the same way that Native Apps have access.

Those that don't seem to care for any of this insistence, tend also to be naive as to the massive differences between TCP and WebSockets, and IP and TCP and the whole stack in general. The WebSocket spec is a good example of people doing things in the most indirect way possible, with a maximum of red tape, as opposed to people doing things in the most direct way possible, with a minimum of red tape.

The Web as we have it in these respects is very much Animal Farm and 1984. There appears to be little thought leadership from the major stakeholders in this regard. People like Tim Berners-Lee are asking for change (http://lists.w3.org/Archives/Public/public-webapps/2012JanMa...), but the new incumbents don't seem to want to see.

milkshakes• 14 years ago

I don't understand the point you're trying to make. Are you against standardization? What specific actions would you take to "minimize red tape"? You do know that TCP, UDP, and POSIX were all developed by committees right? How exactly is the Web in any way like Animal Farm or 1984?

_urga3• 14 years ago

Standardisation for the web targets too much surface area. Browser vendors need to agree on a smaller set of more powerful, more dangerous, more direct APIs so as to put the responsibility for innovation back in the hands of developers. Developers are being wedged further and further away from bare metal, with no right to return, even if they should so wish - this process needs to be reversed.

Websockets 101

Related Stories