Deprecating XML

http://norman.walsh.name/2010/11/17/deprecatingXML

by adambyrtek • 16 years ago

106 91 16 years ago

norman.walsh.name

latch• 16 years ago

an example would be useful

wccrawford• 16 years ago

"XML deals remarkably well with the full richness of unstructured data."

What? No, it doesn't. XML has even more structure than JSON does. Unless you're using it improperly, and the you're negating all the advantages that XML was supposed to bring.

It wasn't for 'mixed content'. It was for system interoperability.

The advantage that XML brings over JSON is that you can have an external document that you can use to validate any XML you're sending or receiving. You can guarantee it's formatted correctly. JSON provides no such guarantee.

Not that that's necessarily a bad thing.

igrekel2• 16 years ago

Think what the author meant is that XML allows documents where the structure is less predefined. It can be the markup that drives the interpretation. With a JSON payload, you need to know how the whole is structured to know how to use it, or build additional mechanisms over it.

Also XML brings other useful features like data types.

Groxx5• 16 years ago

Is there a reason JSON fields can't be an object (of sorts) which defines data type, parsing rules, depth to read, circular structures, etc?

You can convert, losslessly, between XML and JSON. Doesn't that mean they have identical capabilities, though one may be easier in certain circumstances? There's no "need" in any one which the other doesn't also need.

iampims• 16 years ago

There's json-schema[1] if you feel like validating your json payload

[1] http://tools.ietf.org/html/draft-zyp-json-schema-02

Groxx5• 16 years ago

It only provides that guarantee because the parsers have agreed to do so (and not all do, nor do they do so identically). It's a guarantee by communal agreement, nothing more.

There's no reason a JSON format can't do the exact same thing. JSON-RPC, for instance, defines its own rules on top of JSON which anything using it must conform to. How hard would it be to make a set of JSON interchange rules which specify external schema documents?

edit: answer:

  {json_strict_version:1, schema:{external:[url,url], internal:[{schema},{schema}]}}

locopati• 16 years ago

If I'm reading you right, you're saying that XML only provides a guarantee because of an abstract agreement between parsers? Nonsense. XML has a specification for what constitutes a well-formed document and an XML Schema defines rules that assure validity. XML may be verbose and the libraries may be a pain to work with (though there are some well-written ones out there) and there may be other flaws you want to bring up, but this is not a defensible point.

rbetts• 16 years ago

An XML document is mixed content in the sense that it can be validated, signed, partially encrypted (perhaps by multiple actors), can be wrapped by a header, and augmented for application level routing. [edit: where each of these actions may mix-in content with the original document.] And all of this can be done in a standards compliant way.

Now the standards can totally suck - they can be improperly implemented - and they can be difficult to make performant... But those are largely issues of implementation unrelated to the core XML and XML namespace specifications.

It seems unlikely to me that JSON could reasonably be purposed to those use cases...

Udo3• 16 years ago

> It seems unlikely to me that JSON could reasonably be purposed to those use cases...

Since any XML document can in theory be converted into a complete JSON representation I do believe that nothing is keeping people from doing exactly those use cases. You're right in asserting that this isn't happening in practice with JSON. I allege the reason for this is that XML isn't really used for these kinds of operations either. Sure, the XML working group put out a stunning amount of "standards" that prescribe these conventions, but in practice NONE of it actually works across vendors without heavy intervention and some pretty shocking compromises in application code. Most complex XML products are not actually that interoperable, they mostly just work because one or more communication endpoints are basically faking compliance of their interfaces.

jimbokun2• 16 years ago

It really seems that a lot of people in this thread don't get the meaning of "mixed content". That's where you have plain text mixed with tags, and it's difficult to represent with JSON. If you don't think so, I'm curious to see a good way of doing it.

mturmon• 16 years ago

Here's a link that illustrates this concept:

http://seanmcgrath.blogspot.com/2007/01/mixed-content-trying...

It's not as persuasive as it could be, but it gets the idea across.

andybak• 16 years ago

And I'm looking forward to a real world example of 'width' where JSON fails and XML shines.

DanHulton2• 16 years ago

God, but I hate XML, and sad to say, for the same reason a lot of people hate PHP.

I hate XML because it is so mis-used. People try to use it for EVERYTHING and you get these monstrous XML trees with legions of attributes, when what you're trying to send could comfortably expressed in "{success: true, message: 'Record deleted.'}".

I admit my ignorance, at least. I'm sure that were I properly shown well-used XML and these wonderful and vast tools that the competent XML worker has at their disposal, I'd be as happy to work with XML as I am to work with PHP.

That said, you won't ever catch me trying to write a 3D FPS in PHP, and I hope that if you ever DID catch me doing so, you'd give me the exact same fish-slapping that XML abusers deserve.

palish2• 16 years ago

At the game studio I work at, they use XML for all scripting tasks: gameplay scripts and UI scripts.

As in, XML nodes are "game script commands" which are executed sequentially.

Udo3• 16 years ago

That's kind of like XSLT, a language with an XML syntax. I can see how that is an appealing use of XML since it basically is a nice representation of a high-level syntax tree.

I believe you have come up with a rare example where XML is decidedly better suited than JSON!

However, the majority of XML use cases are for document-interchanging applications and the way those are being handled requires massively bloated toolchains of poorly inter-operating software.

vog• 16 years ago

Luckily, embedding more useful languages like Lua, Python or Scheme is becoming more and more common.

DanHulton2• 16 years ago

Dear God.

Why not just... use a scripting language?

I hear LUI is pretty popular as a DSN these days.

DjDarkman• 16 years ago

I find XML to be too heavy and complicated for an interchange format.

Quote: "any damn fool could write a better data interchange format than XML"

prodigal_erik7• 16 years ago

There are hilariously terrible protocol designers out there, and too often nobody with taste manages to stop them. The EDI standards are often cited as examples of how much worse than XML it could get.

http://en.wikipedia.org/wiki/EDIFACT#Example

patio112• 16 years ago

After three years working in Big Freaking Java Web Applications which needed to frequently talk to other Big Freaking Legacy Applications, which is pretty XML's core use case, I struggle to think of a concrete example of a time where XML was a clear win over JSON. I spent a lot more time fighting XML (and, more importantly, libraries built on libraries built on libraries which parsed XML which failed to interact with libraries built on libraries built on libraries which parsed XML in a subtly different and incompatible way) than I spent enjoying the richness of my namespaced mixed content.

We were once, no lie, forced into JSON-over-XML because of technical and organizational imperatives that had to be reconciled with the need to ship functional code sometime this year. ("The web services parser is breaking again!? Screw it. We only need three fields. JSON grab bag, serialize as string, read string and interpret as JSON on other side.")

jimbokun2• 16 years ago

"After three years working in Big Freaking Java Web Applications which needed to frequently talk to other Big Freaking Legacy Applications, which is pretty XML's core use case"

This is absolutely, positively, NOT XML's core use case. This is yet another case of Big Freaking Java Web Application developers finding a way to add gratuitous complexity, along with their dependency injected, aspect oriented, AbstractDataSourceFactoryFactoryImpls.

By the way, a quick Google brought up this, which is evidently not a parody but actual code:

http://www.docjar.org/docs/api/org/outerj/pollo/xmleditor/Fa...

The mixed content case does not occur in the vast majority of Big Freaking Java Web Applications.

patio112• 16 years ago

There are sometimes when the closest route to the goal, given that you are building a Java web application, runs through very small voice a factory factory. They're a possibly degenerate consequence of the interaction of a few things which, separately, defensible ideas.

One of these things is that you want to be able to pass objects over the wire without having to reduplicate all the code on both sides. For example, if you have Student objects in the student management system and also have Student objects in the university SNS, and those systems need to periodically pass data between each other, somebody might say "It would be great if they could do that without us having to custom-write data massaging routines. After all, we already have student code on both systems." and another guy might say "True, true, but we don't want to keep the Student class in sync on both systems neessarily." and then a choice gets made and someone forgets that one of the Students contains a graduation date implemented by a long-forgotten contractor as a wrapper class with some utility functionality which contains a Locale which cannot be serialized and then all of a sudden you are in the sixth layer of XML Hell.

Not that I'm arguing with you -- God willing, I will never, ever have to work with Java architecture astronomy again.

euroclydon• 16 years ago

HTML is pretty much XML, should it be JSON?

xiongchiamiov2• 16 years ago

No, but it's much nicer to use Markdown and friends for a lot of it.

Really, HTML is needlessly verbose and annoying - we're just used to it. If you take a look at something like HAML (not that I agree with all of their choices), you can see how much nicer it could be.

ElbertF• 16 years ago

Try saving a marked up document as JSON.

wvenable2• 16 years ago

You make a good point yet outside of XHTML and perhaps XSLT, I've never seen a document marked up as XML. I have, however, worked on hundreds of projects that used XML as a structured data exchange format, for remote procedure calls, or for configuration.

dfox5• 16 years ago

I routinely save HTML markup as S-expressions (S-XML) which is almost as readable (and slightly easier to parse) as JSON, yet markup in it does not look as weird and ugly.

seanalltogether• 16 years ago

I spend all my time doing front-end development and I'm not sure what the author means by "I look forward to seeing what the JSON folks do when they are asked to develop richer APIs". The only deliberately unstructured XML I've worked with is document formats. Where are these rich unstructured APIs that wouldn't otherwise be classified as lazy?

ilovejson• 16 years ago

I agree XML has it's uses, but I would go for JSON first, XML somewhere after JSON. Any day.

I still love JSON, http://ilovejson.com for it's readability and simplicity.

JonnieCache2• 16 years ago

IMO the biggest advantage of JSON on the web is a simple one: fewer bytes on the wire.

EDIT: also the concise yet all-encompassing nature of its home at http://www.json.org

kenjackson• 16 years ago

Is this really true? It seems like they would be remarkably close in number of bytes on the wire, accept in cases where you generate really poor XML. In fact, in some basic examples I quickly looked at, I think XML actually puts fewer bytes on the wire.

I do have to agree with the author though. This feels like a really "meh" discussions.

xiongchiamiov2• 16 years ago

If the actual data is small compared to the tag names, then XML will be significantly larger.

Goladus• 16 years ago

I have the same reaction as the author, although for perhaps a slightly different reason. XML has far less syntax than JSON, and strings are the default input. So XML is better when you have large chunks of text want to add structure and meaning to abstract subcomponents, and want it to be human-readable. JSON is better when you need to organize lots of little pieces of data.

Unfortunately, neither use case is typical. Usually it's a mix of both, and it's not worth using both formats so you have to pick just one.

haberman5• 16 years ago

XML's bad, JSON is better.

But Protocol Buffers are best. Like JSON, the data model for Protocol Buffers maps nicely onto simple data structures (unlike DOM). But with Protocol Buffers you also get a schema for free, a wicked efficient binary format if you want it, default values, etc.

And you can use JSON as a text format. In other words, you could take your JSON that you have sitting around, whip up a Protocol Buffers .proto file for it, and get nice generated C++ classes for it with full schema validation. Your JSON file:

    {field1="foo", field2=5}

...could be accessed from a C++ object as:

    my_obj.field1();

The only bummer about protocol buffers is that its support for high-level languages (PHP, Perl, Ruby, etc) is not very good. I've been working on a separate implementation of Protocol Buffers in C to address this (by making it easy to write bindings for) but this project has unfortunately been stalled as I've been busy with work and life. :(

alavrik2• 16 years ago

What is it the tool that generates C++ codecs for JSON from .proto specifications?

(I wrote a generic command-line Protobuf-JSON converter).

haberman5• 16 years ago

Hmm, I just looked and apparently this code isn't part of the open-source protobuf release. :( This code exists inside Google, but apparently hasn't been disentangled from the rest of the Google source tree to be released outside.

prodigal_erik7• 16 years ago

The protobuf encoding carries a ton of redundant crap around (if not as much as text formats). You can do a lot better by paying more attention to the schema and omitting facts that aren't allowed to change, like tags on self-delimiting required fields and arrays (ASN.1 PER does this well).

The protobuf tradeoffs only make any sense if you think you can somehow do something useful with messages that are somewhat but not entirely corrupted, because you still have to solve the problem of finding intact field boundaries without being given any HDLC-style framing.

haberman5• 16 years ago

> The protobuf encoding carries a ton of redundant crap around (if not as much as text formats).

[citation needed]

One of the features of protocol buffers is that they're backwards and forwards compatible. You can add and remove fields, change "required" to "optional" and back again, and still make sense of what comes to you on the wire. I don't think there's much of anything that can be eliminated from the protobuf binary format.

Eliminating tags for a field (even if you consider it required) wouldn't be backward compatible with a previous version of the protocol that considered it optional.

A message from a previous version of the schema is not "corrupted." Being able to make wire-compatible changes to the protocol is an extremely important feature.

dfox5• 16 years ago

ASN.1 does not map well to any meaningful and generic in-memory representation (I tried to do that in dfsch's BER/DER module and failed spectacularly), while JSON does, but modifing BER to be efficient json-like representation is not that complex, basic idea is to large extent similar. And as for efficiency and ease of use, if you want efficient dataformat with fixed schema, XDR is certainly better than PER (and free of ASN.1's design-by-comitee nonsense)

sigzero• 16 years ago

These XML vs JSON talks are tiring. They solve entirely different problem domains. XML is for documents and JSON is for data. It is not the fault of the XML spec that programmers go nutso on its usage outside of its problem domain.

j_baker4• 16 years ago

I don't understand your point. Documents are data, are they not?

alavrik2• 16 years ago

Documents are semi-structured data, that is they are a mix of text and structured data elements put into a structured hierarchy.

dfox5• 16 years ago

problem is in that 90% of applications does not need to process mixed-content documents, yet because XML was the new cool-and-interoperable-solution-to-everything it got to be used for anything and mainly for things that are only simple structured data. Probably because you can abuse it for that and as a bonus you get lots of tools (like XSLT)

grovulent• 16 years ago

I'm no expert on these things...

But generally what I want out of a data stream is a set of name/value pairs. That's it. I really don't understand why, but trying to get those name/value pairs from an xml based api is always an adventure. Well - I know why. It's because everyone structures their data differently when they create xml. Sometimes the name I'm after will be a node name. Sometimes it will be a value of a text node. Sometime the value that properly maps to that name will be a textnode 3 layers deep, sometimes it will be an attribute value.

I'm still a newb but I've worked with about 8 different apis now. For the json apis I wrote one recursive function that worked on all of them to get the data I wanted... It was about 8 lines of code. For the XML apis, I STILL haven't figured out how to write a recursive function which works for ONE of them, let alone all of them. (while relying on python minidom to do the actual parsing for me).

plq• 16 years ago

one good property of xml is that it's also a database. it can be queried once inside, say, postgresql's xml type.

json lacks that level of support from mainstream relational databases, which makes it a waste of time for certain use-cases.

simonw2• 16 years ago

http://wiki.postgresql.org/wiki/JSON_datatype_GSoC_2010

b-man• 16 years ago

How about SXML?

http://okmij.org/ftp/Scheme/SXML.html

dstein4• 16 years ago

JSON falls flat when trying to define data types and serialize objects.

  var foo1 = new Foo1();
  foo1.bar1 = new Bar1();
  var foo2 = new Foo2();
  foo2.bar2 = new Bar2();

If you were to serialize that data using XML you might do this:

  <Foo1 name="foo1">
    <Bar1 name="bar1"/>
  </Foo1>
  <Foo2 name="foo2">
    <Bar2 name="bar2"/>
  </Foo2>

But now what if you use JSON? You'll probably end up with something like this:

  {
    "foo1" : {
      "class" : "Foo1",
       "@bar1" : {"class":"Bar1"}
     },
    "foo2" : {
      "class" : "Foo2",
      "@bar2" : {"class":"Bar2"}
    }
  }

Generally it makes sense to use XML for imperative data structures, and use JSON for functional data structures.

j_baker4• 16 years ago

I don't see what you're getting at. First of all, how is example 1 better than example 2? And how does that prove that JSON is better for functional data while XML is better for imperative data?

dstein4• 16 years ago

It boils down to JSON being schemaless. In XML the data type is implicitly defined by the tag, whereas in JSON you need to arbitrarily encode the datatype as a property and somehow differentiate between properties and child elements/objects.

jtchang• 16 years ago

XML just gives me a headache.

I just like to point out that there was a time before XML. Fixed width formats are still very common when working with mainframe systems. Surprisingly, they aren't actually that bad to work with.

chipsy• 16 years ago

I am a huge JSON fan, but I have still managed to find a few cases where I can make use of XML - the mixed-mode situations the author describes.

To generalize my viewpoint: XML is a good way to mix data and data processing, or mash up multiple DSLs in polyglot fashion. When stream-parsed, one can imagine great use-cases of XML, where an early node adds context, references and functionality for other data later in the document. But it sucks for plain old standalone-structured data, which is the kind of data we usually like to _store_.

nradov• 16 years ago

The lack of a standard for handling namespaces is a fatal flaw in JSON for many applications. Some of the healthcare data exchange formats we deal with require mixing multiple XML namespaces together; it's complex, but it works. Trying to do the same in JSON would be a huge mess. http://blogs.sun.com/bblfish/entry/the_limitations_of_json

sielskr• 16 years ago

In various conversations on the internet, starting with comp.text.sgml where XML was born in the 1990s, I have seen many signs that someone hopes or expects some of the immense richness in meaning that humans assign to words and other symbols to be carried along by the XML.

In other words, I've seen many special cases of the mistaken belief that programmers of today can do things that only researchers of the future wielding human-level AI will be able to do -- and that just has to cause grief if those hopes and expectations inform IT decisions.

wvenable2• 16 years ago

I remember when XML was still young and I was introduced to XML-RPC. It was truly beautiful protocol -- straight forward, human readable, human writable, and easy to implement. I wrote desktop apps that connected to web services written in different languages on different platforms and it all worked.

Then SOAP happened. The same philosophy as XML-RPC gone horribly wrong: a human readable format no longer readable by humans, insanely complex, and hard to implement properly (even with the libraries). Frustrating all around.

Although JSON is much simpler than XML, there's no reason why someone couldn't invent something as horrible as SOAP (or XSLT) on top of it. I can only hope that JSON implementers see the value of keeping things simple. Perhaps the existence of XML as an "enterprise" technology will help differentiate JSON in that way.

roel_v5• 16 years ago

I used xmlrpc for this exact purpose (desktop to 'web service' - now there's a buzzword I haven;t heard in 5 years!) early this year, it was awesome. On the server side I had PHP's simple xml api, which also made pretty much all of the pain of parsing xml go away.

johnm• 16 years ago

FWIW, if you change "unstructured" to "semi-structured" then Norm's argument works much better.

est• 16 years ago

I remember a while ago I read a submission on HN (or reddit?) talking that a founder of jabber protocol is ditching XML for something new. Anyone remember?

joseacta• 16 years ago

I think they shouldn't be mixed. JSON has its strong attributes as also does XML. I started working with XML on 1999 and started doing AJAX style web applications on that time using XML/XSL and Internet Explorer XMLHttpRequest. So now we have JSON, good, excellent addition. Is fast and simple to implement.

But to deprecate XML, I don't think so. Imagine just an EDIFACT document in JSON?

Deprecating XML

Related Stories