Publishing JSON over XMPP

http://blog.fanout.io/2013/10/09/publishing-json-over-xmpp/

by jkarneges • 13 years ago

38 17 13 years ago

blog.fanout.io

I wonder when will we have support for JSON transport in XMPP (or JMPP if you can call it that).

ralphm3• 13 years ago

This has been discussed many times, but so far, no clear advantage of changing the wire-protocol, to be based on JSON instead of XML, has been shown. Especially if you want to retain distributed extensibility you will mostly end up with curly brackets instead of angle brackets. Sure, dealing with the DOM in the browser is not awesome, but in general this could be dealt with by having better library APIs instead of changing the wire-protocol.

That said, a minimal protocol addition like this, which just provides an element to wrap serialized JSON for payloads, definitely help with minimizing that DOM pain. I will propose standardizing this as a (very short) XEP (XMPP Extension Protocol) with the XMPP Standards Foundation.

MattJ1003• 13 years ago

That said, a minimal protocol addition like this, which just provides an element to wrap serialized JSON for payloads, definitely help with minimizing that DOM pain. I will propose standardizing this as a (very short) XEP (XMPP Extension Protocol) with the XMPP Standards Foundation.

Someone agrees with me!

I didn't find much support when I did exactly this last year, so I didn't push it forward, but: https://matthewwild.co.uk/uploads/xep-json.html

jkarneges• 13 years ago

Other than jokes, I think the closest we have to something real is XMPP-FTW, which operates as a gateway and JSON-ifies everything between itself and the XMPP server.

Personally, I'd like to see XMPP go format-agnostic, just like HTTP. What's funny is that there was already a competing protocol that did exactly this called BEEP, created around the same time as XMPP. Developers ultimately preferred XMPP, and I'd attribute this partly to the fact that the payload format (XML) was baked into the protocol. Back then, people loved their XML about the same as people love their JSON today.

Interesting how things change in 10 years.

MattJ1003• 13 years ago

As a server developer I'm in a good position to implement this, and every so often (a couple of times a year), someone will ask for it.

There are two approaches that could be taken. The first is to attempt to map XML losslessly to JSON. This does not end well. JSON's simplicity as a data structure is lost, because it doesn't support namespacing, on which XMPP relies heavily. For a fun take on this, but grounded in reality, see XEP-0295: http://xmpp.org/extensions/xep-0295.html

The second approach is to drop namespaces, which means dropping the 'extensible' part, and defining some mappings of common stanzas (XMPP 'packets') to and from JSON. This is what xmpp-ftw aims exactly to do (on the client side): https://xmpp-ftw.jit.su/

I personally think the second way is saner, if you accept that you're closing the door on one of the main features of XMPP provides - interoperability between a diverse set of clients.

Now whether the translation ought to happen on the client or the server side is up for debate, and depends on why you feel you want JSON in the first place. If it's simply to ease your life as a developer, the client is better - just use a high-level library like xmpp-ftw and forget about it. If it's because you have some low-end platform where a streaming/SAX XML parser isn't available, you really do want JSON (or more likely, something even lighter) as the wire protocol.

jerf• 13 years ago

"I personally think the second way is saner, if you accept that you're closing the door on one of the main features of XMPP provides - interoperability between a diverse set of clients."

Might I just add that we are in the context of discussing an article in which that feature was just used. A new namespace was defined by fanout.io and deployed, and it's all perfectly standard and working.

Those who think that XMPP can just drop in JSON don't understand how extensively XMPP uses XML, and probably don't fully understand everything the XML stack adds.

The history of XML in a nutshell is that it was designed for documents, but got used by a lot of people for data. For data, XML was massive overkill. The masses, as usual, overreacted and declared that XML was overcomplicated and bloated and useless. This is not true. It's actually a very nice set of standard functionality to use... when you need it. If you don't need extension namespaces or tag attributes in addition to contents (two dimensions of attributes, basically) or markup in your text, etc etc, don't use XML. On the other hand, if you do need those things, equally strongly, don't use JSON! It turns out XMPP uses all of the above, plus some more. Those who propose switching it to JSON, IMHO, merely demonstrate that they don't understand the tradeoffs in question, and probably have an irrationally-strong aversion to XML.

I, ironically, say this in the context of someone who in my job have almost always called for XML to be replaced by JSON. This is because in those cases, it was the right answer. I suppose, sooner or later, the opposite will happen.

hamidpalo• 13 years ago

Why? You already have it just the way fanout has done it with an extension.

XMPP as a protocol is very throughly thought out and mature with tons of libraries, clients and servers.

junto• 13 years ago

This XML / JSON markup combo looks so dirty:

   <message from="pubsub.fanout.io" to="alice@example.com" type="headline">
     <event xmlns="http://jabber.org/protocol/pubsub#event">
       <items node="/d04fd503/test">
         <item id="3ebc205b-ce77-46c3-843a-49e203e7945b">
           <data xmlns="http://fanout.org/protocol/fpp">
             {"foo": "bar"}
           </data>
         </item>
       </items>
     </event>
   </message>

est• 13 years ago

reminds me of XML islands in Javascript

    <SCRIPT ID="XMLID" LANGUAGE="XML">
      <XMLDATA>
        <DATA>TEXT</DATA>
      </XMLDATA>
    </SCRIPT>

or E4X

    var sales = <sales vendor="John">

ralphm3• 13 years ago

While yes, it may not excel in beauty, when do applications developers really need to look at, or implement, wire-protocols? Embedding JSON inside an XML element seems to be one of the least horrible things you could encounter. Binary protocols or base64 encoded blobs or SDP are arguably worse.

As I mentioned in the other thread, if you want to make XMPP easier for application developers, things usually start with a well-defined API. If you want to, say, provide a simple API for doing publish-subscribe over XMPP, you don't really have to expose the intricacies of the XML-based wire-protocol. Most of the variables in this example (the addresses, the node identifier, the item identifier) can be exposed as simple attributes to a class representing a request, or as parameters to a function that performs the request.

However, normally, the payload format is still expressed as an embedded XML document, like Atom or the various payload formats for User Location, key exchange, etc., as defined in a number of XEPs.

Now, if your application just wants to be able to publish and/or subscribe to data easily expressible in JSON, why not just wrap that in a well-defined XML element, so that you can make your API as developer friendly as possible?

And, if you really want to go all the way, you could even do some server-side translation between XML and JSON payloads so that pure XMPP clients can still play without having to deal with JSON. There are some more advanced features in the XMPP Publish-Subscribe protocol to express such translations.

KaiserPro• 13 years ago

I would think that a nice binary blob (be it messagepack or custom) would be far nicer and more efficient.

dksidana• 13 years ago

I would say, JSON over MQTT makes much more sense.

MQTT (http://mqtt.org/) is certainly not as popular as XMPP but much more simpler.

ralphm3• 13 years ago

Makes much more sense for what, exactly?

There are a number of protocols that have been promoted for use in the Internet of Things arena, for example, including MQTT and XMPP. All of them have different strengths and weaknesses.

Cisco did a nice short overview of them with a minimalistic comparison: http://blogs.cisco.com/ioe/beyond-mqtt-a-cisco-view-on-iot-p.... In the comments there, Peter Wahler goes into some more depth on the advantages that XMPP brings to the table. Among them are stronger support for authentication (SASL) and transport security (TLS and channel binding for some SASL mechanisms) and federation (including connecting firewalled entities). And, of course, an established base of libraries, server implementations and deployment.

Other protocols have their own strengths in certain settings, and I can totally see hybrid solutions. For example, using MQTT for local use and XMPP for bridging networks.

Publishing JSON over XMPP

Related Stories