In Defense of XML

https://blog.frankel.ch/defense-xml/

by tannhaeuser • 6 years ago

136 254 6 years ago

blog.frankel.ch

bloak• 6 years ago

The article compares XML with JSON and YAML. That makes sense for some applications, but for other applications, like a structured text, I'd compare it with Markdown. I'm not happy with XML or Markdown for my source, where "source" means what I edit or what I put under version control, which aren't necessarily the same thing. I find it better to invent my own adhoc notation and write a Perl script to verify it and convert it into XML, HTML, Markdown, JSON, ...

It's a shame, but the sad fact is that humanity does not yet have an adequate and widely used way of representing structured text in a computer. XML is perhaps the closest I've seen, but the tools don't seem to be good enough or sufficiently available, and now XML is also unfashionable so I don't suppose the situation is going to get any better.

tannhaeuser2• 6 years ago

SGML, of which XML is a proper subset, has had custom Wiki syntax parsing (such as for a fragment of markdown or CSV) since 1986. Agree with what you said about pointless comparisons of JSON vs YAML vs XML, though.

Edit (regarding your point about humanity not having an adequate and widely used text format): we have SGML and it's offsprings XML and HTML. We won't be getting any new formats/standards in these times of platform economy and vertical integration, ever

ChrisMarshallNY• 6 years ago

You ever hear of BNF[0]?

Ugh. I used to write BNF parsers for a proprietary email system in the early 1980s. I’d rather bang rusty nails through my kneecaps.

It was a fairly complete notation, though.

[0] https://en.m.wikipedia.org/wiki/Backus–Naur_form

Finnucane• 6 years ago

I'd compare it with Markdown

Which is a pretty terrible comparison. Markdown allows you to add some structural annotation to your text as you write it, without having to type a lot. That's a useful thing. But textual markup systems like TEI and Docbook go way beyond what you can do with Markdown (also, what can be practically managed in a less 'verbose' system like JSON). They also aren't really intended as authoring tools in quite the same way; you don't code them as you work.

well, you might do some hand-editing of your TEI in some specialized academic archival work, but you wouldn't use that kind of coding for writing a new text.

earthboundkid2• 6 years ago

Does anyone know enough about LaTeX/TeX or RTF to compare them to XML here?

wbl• 6 years ago

LaTeX is something very, very different. It's based around macro expansion and oriented toward print, not programmatic transformation.

For example if I want a programmatic table of contents for an XML document it's a simple transformation or DOM extraction exercise. If I then transform to XML-FO that will fill in the numbers. LaTeX does multiple full document renders to obtain auxiliary data for a later render to use.

No XML based system does what LaTeX can do for text rendering. It's not impossible they just haven't bothered. What's more impossible is to match the ease of writing text. LaTeX gets in the way much less for most text.

sergeykish8• 6 years ago

It is not lost. "application/xhtml+xml" accepted by all browsers. Browser DOM is XHTML in disguise

    document.contentType
    //"text/html"
    document.body.namespaceURI
    //"http://www.w3.org/1999/xhtml"

HTML is just a serialization optimized for human authoring, it has its faults

    script = document.createElement('script')
    script.textContent = '</script>'
    script.outerHTML
    //"<script></script></script>"

It may be humanity does not have an adequate and widely used way of plain text authoring of structured text. DOM diff and patch may be lacking as well.

fanf2• 6 years ago

I'm amused because markdown started off with John Gruber inventing his own adhoc notation and writing a Perl script to convert it into HTML ...

nimish• 6 years ago

The major issue with XML is all the garbage built on top of it like SOAP, WS-* and horrors like XMLDSIG.

Used purely as a markup language it's ok but you really don't want to be mucking around with stuff like canonicalization at all if you value your time.

kanox8• 6 years ago

Used purely as a markup language it's ok

But people don't use xml as a markup language; they use it for data interchange and for that purpose it's horrible because it maps very poorly to data structures in most programming language.

paledot• 6 years ago

Bingo. This is why I hate XML. Nothing wrong with the syntax - I hate that I have to massage arrays, dicts, etc. in cryptic ways to generate the XML output, then massage them right back at the other end. And PHP's SimpleXML module, at least, is anything but - lots of iterators, casting objects to string, and "where did my data go?".

guiraldelli4• 6 years ago

[...] and for that purpose it's horrible because it maps very poorly to data structures in most programming language.

I disagree with you.

A XML file is, by definition, a [context-free grammar](https://en.wikipedia.org/wiki/Context-free_grammar), the same type of grammars used to specify programming languages.

I will make it clearer with the examples below:

    /* a simple data structure */
    struct Point {
        int x;
        int y;
    };
    
    <point>
        <int name="x"></int>
        <int name="y"></int>
    </point>
    
    /* a recursive data structure */
    struct Person {
        std::string name;
        std::vector<Person> children;
    };
    
    <person>
        <string name="name"></string>
        <vector type="person">
    
            <person>
                <string name="name"></string>
                <vector type="person">
                </vector>
            </person>
    
            <person>
                <string name="name"></string>
                <vector type="person">
                </vector>
            </person>
    
        </vector>
    </person>

To me, it is immediate to see the correct data structure in both examples.

If I had to write the second one in JSON, for instance, I would lose the "template type-information" of the list (vector).

basscomm2• 6 years ago

But people don't use xml as a markup language;

I do. I use it to build a simple website (well, 'simple' being relative, I guess), and for that it works fairly well.

agumonkey• 6 years ago

xml alone has a few flaws (empty text nodes) but nothing major

the xml everything culture was a faux-progress (basically language design gone wild)

bawolff3• 6 years ago

I recently had to learn about SAML so i read about XML signatures.

That's probably one of the worst concieved security standards i have ever read. It should be in a museum for how not to make security standards.

Honestly though, i don't think its entirely possible to separate XML from the culture around it. XML encourages a certain way of doing things that leads to bad, problematically complex, outcomes.

jaywalk2• 6 years ago

It's truly a shame that SAML has become the "enterprise" standard when OIDC is superior in every way.

nerpderp82• 6 years ago

No need to defend XML. XML stands the test of time. Yaml has become the evil it was meant to replace.

If XML had-had a command line structured editor at the beginning and integrated with JS a little bit better, it wouldn't have ever fall from grace.

It pains me that json doesn't have comments, and that so much energy has been expended basically getting JSON to the tooling and conformance support that XML already had.

Aloha• 6 years ago

I like XML, I hand write a ton of XML at work, I like it because I consider it to have some built in sanity checking, its something that is obvious even when using an text editor if its correct or not.

arethuza3• 6 years ago

Most editors these days will do similar checking of JSON syntax and a lot do checking against JSON Schema as well if you have one handy.

aspaceman• 6 years ago

I also like XML. Getting XML instead of JSON implemented at work is a damn pain in the ass though. I work in GPU research, and the appeal of JSON over XML to a underknowledgable PI is immense. I think it comes from being able to read it pretty easily as a non-technical person.

ptx• 6 years ago

In what way would you have liked to see the JS integration improved?

I was surprised recently to find out that the browser API doesn't provide a streaming parser for XML, but it doesn't do that for JSON either, so that one at least is a draw.

Marazan2• 6 years ago

E4X

nsonha5• 6 years ago

who even cares, just use things that everyone uses. XML can be used in a schemaless manner and json/yml can be used with a schema. Eventually the more compact formats are more popular because people can use them for other things such as object dump and actually read it. And a very big config file can be handwritten and code review without too much hassle. It's not about schema or association with enterprise.

I do miss SOAP though. Why are we still writing plumbing code in 2020. And no grpc isn't the answer there is still plumbing code, what's the point?

coding123• 6 years ago

REST was the answer, because why have a schema define a specific input, when instead it's a lot more fun to make the input come from one of these fun places: Path, Query, Body (Even with a GET - because why not?), Header, AAND let's not forget the Method!

nsonha5• 6 years ago

let's not forget the prescribed verbs that give rise to CRUD APIs

sk5t6• 6 years ago

To play devil's advocate: why should we have all these weird little files knocking about in /etc, when we could query and mutate everything of interest on the system instead via a Management Instrumentation API?

Tainnor9• 6 years ago

I'm a bit annoyed that the author of the article seems to be imply that the move away from XML is just due to hype.

Also the point about the ad-hoc client validation doesn't seem right to me. If the server changes the XML response, the client may be able to fetch the updated schema (assuming the devs didn't forget to publish the update...), but then what? You still have to handle the new or updated fields in code somehow.

My biggest gripes with XML are twofold:

- It's as annoyingly verbose as Java is; I just get lost looking at all the tags (the namespaces don't help).

- The community seems to have this obsession with solving every problem under the sun with more XML and more specifications. I believe, this is wrong. Trying to standardise across the whole range of data interchange and configuration needs is the wrong approach, IMHO. JSON wins here, because it gets out of the way. I do wish JSON had comments and that JSON schema was better (for when you really need it), but nobody ever had an expectation that keys are unique across all JSON documents in existence, something that XML somehow insists is necessary for its elements and attributes.

But what the author is right about is that YAML is an absolute nightmare, and I can't understand why it continues to be so popular. Apart from the dozens of ways of writing booleans or strings, it's also way, way too easy to forget what level of indentation you're currently at, especially if you're navigating large files like Kubernetes or CI configs.

I think more projects (at least larger ones) should adopt their own DSL for configuration. For example, I find the terraform config format much cleaner than any of the YAML stuff.

kanox8• 6 years ago

The biggest advantage of JSON and YAML is that it has a very straight-forward object model consisting of literals, lists and dicts. All of these constructs map very easily to native data structures in most languages: you can just "load" and "save" an arbitrary json or yaml document.

The XML DOM is very different and does not map in an obvious way to objects in any programming language: you either use a DomDocument or object serialization for a subset of XML.

This alien object model makes programming with XML painful and verbose for no gain.

Sure: standards for xml schema and xpath are pretty nice but there are json equivalents which don't have to deal with elements and attributes. If standardization around YAML and XML was equally good would anyone pick XML?

way too easy to forget what level of indentation you're currently at

This is easily solvable by tools. How is it worse than getting lost in braces or open/close tags?

I think more projects (at least larger ones) should adopt their own DSL for configuration.

Please don't. JSON, YAML and XML config files can be easily and safely manipulated in all programming languages, for custom DSLs people end up using brittle regexps. It's not much better than putting config consts in a code file.

Tainnor9• 6 years ago

This is easily solvable by tools.

Maybe, but I want it to work with standard tools. Most editors seem to arbitrarily jump back a couple of indentation levels e.g. if you add newlines somewhere. This obviously doesn't happen with JSON, since the editor won't have to guess whether you wanted to de-dent or not, it's all explicit. And I'm not making this problem up, it's literally a problem almost every time I have to edit some larger YAML file.

I'm not a fan of whitespace sensitivity in programming languages either, but at least Python etc. make a habit of discouraging 15 layers of indentation.

Please don't. JSON, YAML and XML config files can be easily and safely manipulated in all programming languages, for custom DSLs people end up using brittle regexps. It's not much better than putting config consts in a code file.

What? Why would you parse a DSL with regex? I'm not saying people should write crappy DSLs. You can either write your own proper parser or you just use an embedded DSL, since there are multiple languages out there who are quite good at providing DSL support. The bonus is that it's often even type-safe, without having to be as mind-numbingly obtuse as XML.

ithkuil2• 6 years ago

Sure: standards for xml schema and xpath are pretty nice but there are json equivalents which don't have to deal with elements and attributes.

Not quite there yet. Jsonpath is still not standardized (although a colleague of mine is working on it) and the many implementations out there disagree with each other.

(That said, I do agree with the general sentiment against XML)

GoblinSlayer5• 6 years ago

There's DomDocument for json: https://docs.microsoft.com/en-us/dotnet/api/system.text.json...

For yaml: https://github.com/jbeder/yaml-cpp/blob/master/docs/Tutorial...

barefootliam• 6 years ago

"The biggest advantage of JSON and YAML" should be put in a context of, "when working as a programmer with config files".

When we designed XML, we were working on things like a million pages of Unix documentation, or the flight manuals for aircraft, or research journal articles, or transcriptions of inscriptions found on fragments of ancient Greek pottery - lots of small similar documents, or very complex documents.

XML lets people who are not primarily programmers do sophisticated text processing and information design without depending on outside resources.

People who complain about XML being brittle are saying they want a system that forgives syntax errors, in the same way that JavaScript... doesn't... or that JAVA... doesn't... or that C... doesn't...or that Python... doesn't.

XML is unbeaten for the things it's good at. It was (still is, often) a problem that people used if in places where it wasn't so good. But that doesn't make it bad - it means you need to understand where it's good and where it isn't.

AnimalMuppet3• 6 years ago

To me, one of the virtues of XML is that, if the server changes the XML response, you can capture it, and a human can eyeball parse it and figure out what went wrong, and figure out how to change the code on the receiving end to compensate. It's hard to automate, sure, but it's far better than having the server hand you some updated binary format.

Annoyingly verbose: Yup. Absolutely. I thought XML was kind of OK. Then I learned some about S expressions, and looked at XML with those eyes, and it was absolutely disgustingly verbose.

Tainnor9• 6 years ago

To me, one of the virtues of XML is that, if the server changes the XML response, you can capture it, and a human can eyeball parse it and figure out what went wrong, and figure out how to change the code on the receiving end to compensate. It's hard to automate, sure, but it's far better than having the server hand you some updated binary format.

Maybe I'm misunderstanding, but it seems to me you can do the same thing with JSON.

Phrodo_00• 6 years ago

My biggest gripe with XML has more to do with how data is structured: XML offers both tag attributes and child tags and when encoding data it's not obvious which one to use. You can also use multiple child tags and it's not obvious whether you should mix multiple kinds or not.

The usage is obvious when using XML as markup, but when used for data it's left to the one creating the particular spec.

You can have style guidelines, but other data formats don't really need that to answer those questions. More flexible is not always better, especially in languages.

thesuperbigfrog2• 6 years ago

> XML offers both tag attributes and child tags and when encoding data it's not obvious which one to use.
> The usage is obvious when using XML as markup, but when used for data it's left to the one creating the particular spec.

Either approach is valid. Having a choice about how to structure the data is not a bad thing.

My preferred approach is to use attributes for simple key-value pairs that are mainly used to select structured data found in child tags. If you have configuration data that is stored as XML, you could use attibutes to mark where the contained structured data should be used. For example: 1) language localization data could be stored stored in XML with attributes describing the language or locale; 2) build configuration settings might be stored in XML with attributes describing the target platform or CPU architecture.

For more formalized data formats, create an XML Schema that defines what the XML data should look like and then you can easily validate that the XML data is structured correctly and even generate parsers or XML to programming language bindings.

arethuza3• 6 years ago

"it's not obvious which one to use"

Perhaps this explains one system I saw that included entire encoded XML documents as attributes in other documents. On the plus side this embedding only went one level deep.

HelloNurse• 6 years ago

Child elements are different from attributes because elements can be structured (with attributes, other elements etc.) and repeated within the same parent element. On the down side, elements are a little more verbose and whitespace needs some attention.

This restricts attributes to zero or one unstructured values, which is more than a "style guideline".

sergeykish8• 6 years ago

It is questionable even in markup. Attribute restrictions (unique, can't be structured) gave rise to minilanguages

    class="foo bar"
    lang="en-US"
    style="margin:0;padding:0"

XSLT blows away promise of structured data with XPath and functions like concat in attributes. And it is a special case in DOM and XSLT making it much harder to manipulate.

Meanwhile <script> and <style> content hidden by default.

Mikhail_Edoshin3• 6 years ago

There's no ambiguity at all. Attributes are what fields are to a record; basically everything goes into attributes. Child elements are other records and are used when either there's more than one or their order is important or they also appear as children of other elements. Text is only used when there is actual content for humans that XML delineates; when XML is used as "application/xml" :) data there should be no text (unless parts of your data are such text).

bryanrasmussen• 6 years ago

- It's as annoyingly verbose as Java is; I just get lost looking at all the tags (the namespaces don't help)

I guess if you take the effort to make it as annoyingly verbose it is, but in regards to namespaces for example Joe English's email on the psychopathologies of namespace usage http://www.flightlab.com/~joe/sgml/sanity.txt if followed for best practices also has the side effect of reducing much potential verbosity.

solatic• 6 years ago

Also the point about the ad-hoc client validation doesn't seem right to me. If the server changes the XML response, the client may be able to fetch the updated schema (assuming the devs didn't forget to publish the update...), but then what? You still have to handle the new or updated fields in code somehow.

To be fair, there are often tools that will generate object definitions in the target language from an XML schema. It's not a perfect solution - you need to rebuild the project when upstream changes - but it's not like this hasn't been addressed. For what it's worth, Protobuf has the same property.

Tainnor9• 6 years ago

Yes. And then you end up with a domain-model that is badly designed because XML is not capable of expressing rich and nice-to-use data structures (as is possible both in OOP and in FP languages). You end up with lowest-common-denominator stuff, at which point you might just as well use JSON and parse it into something type-safe yourself (or not, and just use it as a [String: Any] dict).

GoblinSlayer5• 6 years ago

Matrix protocol uses JSON with namespaces, both Java-style, C and XML-style namespaces.

SamWhited4• 6 years ago

And because it's non-standard it's irritating. Every JSON parser doesn't have this built in, so you have to implement it at the application layer if you want it.

quercusa2• 6 years ago

- It's as annoyingly verbose as Java is

I reviewed an XML-based international industry 'standard' that allowed both verbose and terse versions of the same tags. Example files I saw tended to use the terse ones for common tags and the verbose ones for rare ones.

SahAssar4• 6 years ago

The last time I dealt with XML based services I tried to push "If we can't be right let's at least be consistent"

Multicomp• 6 years ago

I really would prefer to use XML for my next project, but self-censored myself so that if this project is ever open source and has other contributors, I'm not instantly dealing with hordes of users who want to convert it to json for data transfer objects.

Maybe it is because my baby duck syndrome came with xml web services and asp.net web forms, but it is amazing how much scaffolding and engineering work we threw away and are now reinventing. (Wsdl, schemas, etc)

dnautics2• 6 years ago

Maybe I don't understand something about xml. As in, best practices. When designing an xml schema, When are you supposed to use innertext, when are you supposed to use a tag inside your tag, and when are you supposed to use a tag property?

xapata• 6 years ago

First, consider whether someone will ever want to convert to JSON. If so, make that easy.

ZeroClickOk• 6 years ago

Mag property is mostly for metadata, but it does not stop it to use it for this purpose. All three options are valid and you can use as will, xml is flexible enough in this point (unless your xml consumer has some restriction, then follow the consumer spec)

roel_v• 6 years ago

Quite often it just doesn't matter that much. It's a matter of 'taste', as are so many things that are 'designed'. And yes you'll make mistakes a few times. The correct response to those mistakes is to take them as learning opportunities and not dwell on them, instead of blaming and trying to replace the tools.

Of course one can use this exact argument to stave off any change ever...

Kednicma• 6 years ago

The main problem with XML is that writing a safe XML parser is difficult. But writing, say, a safe JSON parser is not much easier; writing a safe ASN.1 parser is much harder. So it's not clear whether there's a better possible declarative format.

aidenn0• 6 years ago

Writing a safe tagged netstring parser is pretty darn easy, for many definitions of safe.

There's always the issue of very deeply nested data structures causing lots of allocations, but no nested parser can avoid that.

Full featured XML parsers can read arbitrary files from disk, which is usually not what you want from a data interchange format.

AnimalMuppet3• 6 years ago

Gah. I dislike XML, but I had forgotten how much I loathe ASN.1.

earthboundkid2• 6 years ago

XML is hard to parse because they mixed compression concerns in with encoding, which is a security nightmare. What’s hard about JSON? There was some looseness around the spec for certain characters, but overall, it’s pretty straightforward.

tantalor2• 6 years ago

JSON is JavaScript native, while XML is not

Yet we have XMLHttpRequest to thank for the last 20 years of the web.

jaywalk2• 6 years ago

It is pretty ironic, although XHR does predate JSON.

tantalor2• 6 years ago

Crockford says JSON existed since JS was formalized.

He says he discovered it in 1999 but it was in use Netscape as early as 1996.

emmanueloga_• 6 years ago

XML specified with a mix of RelaxNG and Schematron is a really nice pairing. Both are interesting and a little different to the usual schema languages.

RelaxNG is like "regular expressions for trees". It has a compact syntax like this:

    start = element library {
      element book {
        attribute title { text },
        element page { text }+
      }
    }

.. so books in this library (root element) should have a title attribute and one or more pages. There's a (pretty geeky [] :-p) description of the algorithm here [1] (: Brzozowski derivatives + Haskell!).

Schematron is another schema language that allows specifying constraints as a series of pattern rules:

    <schema xmlns="http://purl.oclc.org/dsdl/schematron">
      <pattern>
        <rule context="book">
          <assert test="let $b := .[starts-with(@title, 'A')] return $b/page/count() >= 100">
            Books whose title starts with "A" must have 100 or more pages.
          </assert>
        </rule>
      </pattern>
    </schema>

So it allows specifying constraints that would be too tedious or impossible with RelaxNG, and allows keeping the RelaxNG grammar simple.

The test on the Schematron "assert" element in the example uses "let expressions" [2], one of many additions that makes XPath and XSLT 3.0 feel a lot like any other modern functional programming language (other features: higher order functions, maps/arrays/JSON support, and more). This presentation [3] shows a few more 3.0 features.

If lisp is for lists, then XSLT definitely is for trees! I suspect as people start forgetting the baggage of SOAP and XSLT 1.0, the language is going to become more and more popular since it is so useful. BTW, there's now a JavaScript implementation that runs on node and the browser too [4].

1: https://relaxng.org/jclark/derivative.html

2: https://www.w3.org/TR/xpath-30/#id-let-expressions

3: https://www.kosek.cz/xml/2019xmlss/Kosek_XSLT4DailyCoding.pd...

4: https://www.npmjs.com/package/saxon-js

tialaramex2• 6 years ago

One reason to dislike XML is that it can't actually encode text.

This is deliberate, key people involved in XML standardization really hate some of the characters (particularly some "control characters") in text, they don't think those characters should exist, so, they outlawed them in XML.

I know what you're thinking, "Just escape them". Nope. I didn't say you have to escape them, I said they're outlawed, the XML definition says these characters, whether escaped or otherwise, must not appear in an XML document and some XML libraries enforce this requirement from the standard.

Now of course if you know this will happen and you want to sidestep it you can encode all your text in some other way (e.g. as Base64) and then use XML, but wait a minute, the whole point of XML is that it's a text encoding, for encoding text, so this is crazy.

In systems I've built that are mandated to emit XML and yet are expected to handle text anyway, I patiently explain the problem and unless people seem stirred awake and back off on requiring XML, I then convert all prohibited characters to U+FFFD - the Unicode replacement character - and they can keep both halves of the resulting broken garbage.

saurik• 6 years ago

A "control character" (such as form feed) is kind of by definition not "text", but more importantly this was fixed for XML 1.1, which is like 14 years old at this point.

matharmin• 6 years ago

I have never seen XML 1.1 used in practice. How many parsers actually support it?

sk5t6• 6 years ago

Precisely which (printable?) characters do you understand to be outlawed in cdata/pcdata by what XML standard?

For what it's worth, I'd not take a charitable view of this bold assertion in the general case.

mrbonner• 6 years ago

You heard of CDATA, bro?

ithkuil2• 6 years ago

Standard says: "

CData ::= (Char* - (Char* ']]>' Char))

Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] / any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */

Which means the following characters are illegal: #x0, #x1, #x2, #x4, #x5, #x6, #x8, #xB #xE, #x18, #x19, #x1A, and #x1C

I'm not arguing here for or against inclusion of these characters as legal characters in a text block; just saying that GP's point is oblivious to actual encoding (explicit escapes or cdata)

archgoon• 6 years ago

This is an odd criticism. JSON, the popular alternative, also has this 'issue'. In fact, almost any non binary solution, like protobufs, will have this issue, as it's a text, not byte, encoding. The reason for outlawing these characters is that they aren't text, they are, as the name implies, control characters.

tialaramex2• 6 years ago

Unlike XML you can simply escape control characters in JSON and "almost any non binary solution". Prohibiting this is a weird misfeature of XML.

throwaway_pdp092• 6 years ago

Of course it can't. It's designed to be human-readable which control characters aren't. What eg. does BEL (ascii 07 IIRC) look like do you reckon.

bawolff3• 6 years ago

Ideally it would look like 

Its useful to be able to encode arbitrary characters even if they aren't human readable.

butterisgood• 6 years ago

In the early 2000s XML was like violence - if you didn’t get what you wanted out of it you just used more.... or some horrible analogy like that.

TOMDM• 6 years ago

I feel like a horrible analogy is itself the perfect analogy for XML.

dragonsh• 6 years ago

Reason XML went out of favor I think has to do with decision to focus more on efficient parsing by machine and less on readability and simplicity. There was a competition between XSD (XML Schema Definition) [0] and RelxNG [1] for defining the structure of XML. Personally I preferred RelaxNG, but at that time Microsoft and many big players converged on XSD. So XSD won but XML is out of fashion.

I wrote a XML encoder and decoder with XSD for an EDI application handling 2-3GB files in C++ using MSXML and it was very slow, improved the speed using stream parser, but still slow. Later on using libxml rewrote the parser in C with RelaxNG and the processing time reduced from hours to seconds on the same hardware. At that time as RelaxNG lost in spite of having elegant structure compared to XSD. I had a lot of arguments with my friend that RelaxNG will win because it's readable, simple and support every use case supported by XSD. But due to support from many vendors especially Microsoft (as most enterprises were using microsoft) an inferior solution like XSD won, because Microsoft focused more on computer consumption of format instead of human. So although XSD can be read no one could understand it. This is one of the reason JSON, YAML and TOML have picked up because they offer the readability and parsing at the same time. Hopefully people can understand it this time and do not destroy JSON with complex JSON Schema definitions.

Take a look at both:

   Relax NG example in compact syntax:
     element book {
       element page { text }+
      }

   Relax NG exmaple in non compact syntax:
     <element name="book" xmlns="http://relaxng.org/ns/structure/1.0">
       <oneOrMore>
         <element name="page">
          <text/>
         </element>
       </oneOrMore>
     </element>

    -------------
    XSD example:
    
     <xs:element name="book">
       <xs:complexType>
         <xs:simpleContent>
           <xs:extension base="xs:string">
             <xs:attribute name="page" type="xs:string" maxOccurs="unbounded"/>
           </xs:extension>
         </xs:simpleContent>
        </xs:complexType>
       </xs:element>

[0] https://www.w3.org/TR/2008/REC-xml-20081126/

[1] https://relaxng.org/

hnick• 6 years ago

There's also the question of using a SAX or DOM based parser.

More than once, I've had to update a program using 32-bit perl from DOM (simple, fast) to SAX (slower but more control) as documents grew and the scripts crashed due to Out of Memory errors. Using SAX gave us event based handling to ignore certain nodes, or cache them to disk in another format as they got too big, and other fun stuff.

Rule of thumb was to multiply the document file size by 10 to get the memory footprint with a DOM approach, because of all the objects, relationships, and metadata that has to be stored for everything. One individual records got up above 100MB on disk we were in the danger zone.

Whereas JSON just loads into hashes and arrays and tends to not pose similar problems.

tannhaeuser2• 6 years ago

There's a bit more to it, though. XSD keeps backward-compat with XML DTDs, and XML DTDs, in turn, keep compatibility with full SGML DTDs (XML being specified as an SGML subset). Now DTDs are usually required in SGML unless you're using XML-style canonical markup without omitted tags and attribute names. What makes them required in regular SGML is that element declarations carry tag omission indicators instructing an SGML parser to infer contextually required tags. Tag inference in SGML is deterministic; that is, the parser only considers start-element tags that are required and end-element tags for definitely completed content models at the context position. Also, in the presence of tag omission, an SGML DTD is required even for parsing (for example, HTML cannot be parsed using generic XML rules but needs element-specific content and tag omission rules).

So the design rationale for XSDs to retain backward-compat (what's responsible for XSD's "unique particle attribution" constraint) leads to a different result compared to RNG. By contrast, RNG is a "clean restart" working only for canonical XML-style markup, and is always optional (and only used for "validation").

One could argue XML took away what makes SGML a true authoring format to result in only a canonical delivery format, but these were willful design choices when XML was subset from SGML.

Now as to whether XSD's or RNG's format for grammars are prettier is a moot point. IMO there was never a reason for using an alternate XML syntax for SGML/XML DTD regular expressions/regular content models. Eg compare your example with it's equivalent (simple, compact) traditional element declaration:

    <!element book - - page+>

bawolff3• 6 years ago

less on readability and simplicity

I strongly disagree. The reason XML went out of style is because it in practise was a convoluted complex mess that was difficult for both humans and computers to read.

arethuza3• 6 years ago

I like XML (though rarely use it these days) and JSON but I would have to say that the worst XML I've seen was far far worse than the worst JSON.

dec0dedab0de• 6 years ago

My biggest problem with XML was using APIs written in Java where it seemed like all the internal classes were just dumped however they came out.

I think the simplicity of JSON forced people to think about how to serialize their data. This Simple serialization made things easy for multiple languages, and therefore it became popular.

acdha4• 6 years ago

I don't disagree with every point the author makes — and I cry inside a bit every time I see YAML used for serious work, with data loss and outages caused by that ambiguous — but I think it'd be useful to focus on why XML became unpopular. Java and SOAP culture had a role there but most of it came down to two things:

1. The user experience sucked

2. The developer experience sucked

What XML needed was good quality editors, validators, programing language libraries, and, above all, documentation for people who aren't on the standards committees. What we got instead were people on standards committees and enterprise architecture groups trying to show off what deep thinkers they were, with the assumed inevitability meaning that boring details like actually implementing something could be left as an exercise for the student.

Namespaces are a great example: this is a core language feature, a really powerful tool, and you'll hit the laziest possible implementations as soon as you try to use them. You start with something reasonable:

    <foo xmlns="http://example.org" xmlns:custom="http://example.org/somethingelse">
        <bar>
            <custom:baaz>…</custom:baaz>
        </bar>
    </foo>

That's a nice way of separating ownership, looks great. Then you start trying to parse it, write XSLT or XPath, etc. and find that you can almost never write “custom:baaz" the way it shows up in the document, or often even /foo/bar, because the tools expect you to work with the internal representation (“{http://example.org}foo/{http://example.org}bar/{http://examp...) and so you might see error messages saying "<custom:baaz> not found” in a document which has that exact string, validation errors from tools which use different libraries than what was used to author the document, or, better, silent data loss because you didn't notice that some selectors weren't matching. Many programming language interfaces will require you to repeat those declarations everywhere because, of course, it would have been far too much to expect that a query against element <bar> inherit the namespace declaration from its parent.

Again, this is a key feature which has been around since 1999 and decades later you can still still find people writing things like //*[name()='baaz'] or stripping namespaces entirely before rage-quitting XML. Everything harder is worse: basic usage requires internalizing 6 different ponderous specs on different release cycles, figuring out which of the examples had never been tested and which worked when the author wrote them but are now unsupported by the particular tools you're dealing with, etc. Bonus points for realizing that you had a never-valid document which was just punted down the line by multiple enterprise systems which have had increasingly lenient validation added over the years by developers who were told it was too expensive to fix the previous system.

I'm not sure XML can be saved at this point but I'd start by modernizing libxml2 since half the open source world is stuck in stasis when that stopped getting new features in the early 2000s, and then focusing on non-expert usage. A full GUI editor would be a lot of work but simply having high-quality command-line tools and implementations for common programming languages would go a long way towards not giving people excuses to switch.

jandrewrogers• 6 years ago

My reasoned objection to XML, having used it for many years, is that it is unconscionably expensive to parse. At scale, XML is extremely computationally expensive in almost every dimension. It is not a good way to represent almost any non-trivial data structure, but that is frequently its sole use case. XML is a great way to expand your carbon footprint in the data center.

Today, there are very, very few cases where I would recommend XML despite it being widely specified for standards.

zaphar2• 6 years ago

While I don't usually reach for XML when I need a data transport format this particular criticism is somewhat unwarranted. SAX parsing of XML is pretty cheap computationally. The problem is cultural more so than technical. Pretty much every language ships with a DOM parser. Not all of them ship with a SAX parser. As a result DOM is the more predictably available choice.

DOM Parsing is both computationally and memory expensive though which I think is where XML gets this reputation from.

rjkennedy982• 6 years ago

SAX parsing of XML is pretty cheap computationally.

SAX parsing is also very complex programatically. Who wants to switch on various SAX events when handling a XML document? No, we want the full thing parsed into an object.

I've spent a decent amount of time in the XML world and I've only seen people use SAX when performance was a large concern or when parsing enormous XML documents.

Tainnor9• 6 years ago

Just as an anecdote, I recently had to generate some XLSX which, even for XML standards, is a horrible standard and the only library that seems to be able to accomplish this (for Java) consists basically of some autogenerated classes from the XML schema itself that have zero documentation.

But the worst part was when I tried to add an integration test for it. I tried comparing the generated file with a fixture file, but that didn't work, because XLSX are zipped XML files and zip isn't deterministic. But even comparing the two file directories of the unzipped files and then comparing XML files pairwise didn't really work, because apparently, XML can be serialized in different ways and it turned out that my CI somehow serialized it differently than my machine (somehow the namespaces were handled differently - not sure if it was an OS issue or some transitive dependency issue). In the end I was only able to solve it by using another library that can specifically diff XML files.

virtue3• 6 years ago

if you smash the file permissions and dates on the file it can be deterministic.

Tainnor9• 6 years ago

Maybe (didn't look into that), but that wouldn't have solved the "somehow, XML is serialized differently in some cases because of namespaces" issue which, honestly, I didn't even try to understand because it seemed like a pointless rabbit-hole.

Mikhail_Edoshin3• 6 years ago

Nowadays there is a canonical serialization for XML. Not sure if all tools support it though.

deckeraa3• 6 years ago

One thing not mentioned in the article is the lossy nature of XML when coming from data structures with concepts of arrays and maps. A typical array in XML looks something like:

<foo>

  <bar>First element</bar>

  <bar>Second element</bar>

</foo>

which can lead to issues if the second element isn't in there: should bar just be a property of foo or should it be an array? (I'm definitely open to correction on this if someone knows a better way; my experience with this particular issue is from writing a parser to take XML and turn it into a MUMPS-native data structure. I mostly had to have the parser take an educated guess and then let API users specify which elements were arrays if they wanted it to parse the same way against all inputs).

This is analogous to how saving Clojure sets (e.g. #{:a :b :c}) to a JSON document in CouchDB is also a lossy operation because JSON doesn't have sets, just maps and arrays.

cmwelsh• 6 years ago

Seems like you’re more interested in encoding your implementation details into a text document. This seems like something that could be left to the parser. Provide the target types for the deserialization. This isn’t Python pickle or Java serialization - it’s a text string.

deckeraa3• 6 years ago

Fair enough. Though wouldn't that work better with languages like Java where the programmer typically ends up making a type or class for most things?

With languages like Clojure it's fairly typical to declare no classes or types and to simply have your data structure be a composition of maps, sets, and vectors, etc.

I suppose you could use your Clojure.spec keywords as the tag names in XML, but that's probably going to be more work than (clojure.edn/read-string (slurp "my-data.edn")). Could be an interesting option if you had to use XML as an interchange format.

rcxdude• 6 years ago

Problem is XML is extremely frequently used to represent such types (probably even more so than it's used to represent marked up text). And it's not very good at it.

sk5t6• 6 years ago

It's the same problem you'd get by looking at a JSON document out of context ("is this array-like thing an array, list, set? ordered or unordered? type bounds on members? are these integers or decimals with no remainder?") You gotta refer to a schema document or human-readable document or whatever and not guess.

deckeraa3• 6 years ago

I'd agree that things like JSON and EDN don't convey all the type information that you might want (things you mention like type bounds, orderedness, number format). Though at least in a JSON string like {foo: [elementOne elementTwo]} you know that foo is a list of things, which is something that the XML document doesn't encode for you.

So perhaps I should make my statement more precise and say that XML data, in the absence of the schema information, is lossy compared to some formats like JSON or EDN.

[Edit: to answer the potential question of why I'd be using XML without have schema information, I was being provided the XML by an API that called web APIs and converted the result of the web call into XML (even if it was JSON) before passing it to me.]

lmm4• 6 years ago

IMHO, this argument is shallow, as all 3 formats are text-based. Thus, you can - and should - compress files. Parsing might be a bit slower, but it depends a lot on the exact parser (and the associated technology stack). In the end, the overhead of transmitting and parsing in XML - if any - is negligible compared to the total time in the whole use-case.

That doesn't help a human reader. The XML overhead doesn't matter to computers, agreed, but it matters a lot to people.

Composable While XML doesn’t strictly enforces namespaces, it’s considered a good practice. This way, similarly-named entities defined in different namespaces can co-exist in the same document without confusion about semantics.

Sounds useful but it actually isn't. JSON is composable just fine: just put a JSON document as a value in another JSON document (something you can't actually do in XML, you have to strip the headers). By contrast, XML namespaces are this hugely complicated non-solution to a problem that no-one ever actually has: if you write an XPath expression then it might select something inside an embedded XML document instead of the thing you were actually looking for, because the embedded document might use the same tag name. So to protect you from this bug that has never happened, you can't actually write an XPath using the tag names you see in the document, you have to either expand out all the namespaces to their URLs (and good luck with what your XPaths look like now), or somehow "register" all your namespaces with your parser before you parse. It's just stupid overengineered nonsense.

Several XML grammar implementations are available: Document Type Definition, XML Schema, Relax NG, etc. The most widespread one is XML Schema. Since a XML Schema is also written in XML format, a web server can host it. Then you can reference it by a publicly-accessible URL.
This approach solves the above issues: when a client receives an XML document, the former looks at the XML Schema URL. It can then fetch it, and check that the data conforms to the schema.
Changing the data format is as simple as versioning the XML Schema file, and publishing it under a new URL.

And then when your webserver goes down, everyone's programs stop working because they can't load your schema.

Has XML schema ever actually done anything useful for anyone?

sergeykish8• 6 years ago

Once I've joined a project and there was a problem with text selection in embedded PDF.js

It took some time to find out that CSS rules clashed with PDF.js invisible markup that provides text for selection on top of <canvas> rendered document.

And I am not the only one [1]. Solution? Embed namespace in class name [2] like it is C - gtk_window_new, glBegin, XOpenDisplay.

Yes, namespaces in XPath looks awful, entire plain text XML authoring is awful. XML is application format (hint application/xml), maybe it should not be touched without XML editor.

[1] https://www.google.com/search?q=css+name+collision

[2] http://getbem.com/introduction/

lmm4• 6 years ago

It took some time to find out that CSS rules clashed with PDF.js invisible markup that provides text for selection on top of <canvas> rendered document.
And I am not the only one [1]. Solution? Embed namespace in class name [2] like it is C - gtk_window_new, glBegin, XOpenDisplay.

OK, that's a good example. Though I'd argue that in structured data like XML it's much less likely to happen - you wouldn't generally use an xpath that was just "all tags of the given name", whereas applying styling to all cases of a given CSS class anywhere in the document is relatively common. In any case, the cure is far worse than the disease.

Yes, namespaces in XPath looks awful, entire plain text XML authoring is awful. XML is application format (hint application/xml), maybe it should not be touched without XML editor.

But in that case why bother with a textual format with all the downsides thereof? Something like Thrift or Protocol Buffers makes for much better application formats.

austincheney• 6 years ago

There are a couple of things this article gets seriously wrong.

* Front-end. XML is perhaps more compatible to that front end than JSON because an XML document can make use of CSS and JavaScript where JSON cannot. See this example of a language I created in another life and please that document is literally the XSD schema file: http://mailmarkup.org/mail-documentation.xsd

Also, if you want to use either format in the front end you have exactly the same one step: parsing whether it’s using JSON.parse or DOMParser(). Actually, the browser preferences XML here too because the browser can be told to parse the XMLHttpResponse response and thereby return DOM document object. https://developer.mozilla.org/en-US/docs/Web/API/DOMParser

* Failed for two reasons and enterprise is not one of them. XML has some super intelligent features and technologies. The more intelligently you use it the more powerful it becomes. Most people were not using it intelligently. Most people were using it in the most primitive way possible without any advanced features. In that case it was just a bulky text wrapper. Most of the Java configurations were dumb like that.

XML is big. The charm of JSON is that the syntax is tiny. When pushing data over the internet tiny is better. The only way to make up for the syntax bloat was to do more intelligent and powerful things which XML wanted to do but most users refused to figure out.

fulafel• 6 years ago

Interesting that nobody pointed out the tooling dimension of XML's demise yet. (edit: actually there was another comment too: https://news.ycombinator.com/item?id=24623696)

You really need a high level of support in editors and other tools to make it good for humans to work with, to just showing the AST and to hide the unusably verbose surface syntax. This support didn't materialize widely enough for whatever reason.

Of course you can say that it's just a convoluted way of saying the syntax wasn't good enough - but at the time the argument was that tooling will be good enougg and will solve the issue. So it's still a good lesson about tooling.

wodenokoto• 6 years ago

YAML is to config syntaxes what Python is to programming languages. Seriously, significant whitespaces?

That's his critique of YAML? That it is too pythonic? You're not getting far in convincing people of something being bad, if your top argument is "It is like python".

franga20002• 6 years ago

It sounds like they dislike the fact that indentation matters. I see a lot of people who get mad about that in Python and YAML has the added annoyance of not allowing tabs.

Personally, I love both Python and YAML for this exact reason (among others). Explaining a hierarchical system using indentation to an average person isn't too difficult, but even very smart non-tech people will get confused by explicitly opened and closed blocks (if-endif or { }). Without proper indentation, the code is unreadable anyway, so why not actually take advantage of that and get rid of the bracket soup that means nothing to you anyways.

tasogare2• 6 years ago

It’s not the concept of indentation that’s the problem, it’s the implementation. Tabs vs space, smart editors, auto-linters all make it easier or more difficult depending on the situation, which created insecurities by constantly breaking expectations. In case of code, the fact there is not visible difference (most of the time) between tabs and space make it quite easy for bugs to creep in.

flohofwoe• 6 years ago

A valid criticism of YAML is that it requires a very complex parser compared to XML or JSON. E.g. there's plenty of JSON and XML parsers to choose from for each language, but either none or only one YAML parser, and those add a non-trivial complexity to projects.

(XML and JSON are in the same bucket for me, both are simple to create and simple to parse, YAML on the other hand...)

cpuguy83• 6 years ago

How is xml simple to parse? It is difficult for both humans and machines. Just because there are libraries doesn't mean it is simple. It's a minefield, with security implications.

To be fair... so is yaml.

bishalb• 6 years ago

That's a pretty valid criticism. I am wary of significant whitespaces too as are many people(and one of the reasons I use Ruby for scripting rather than python.)

Xelbair• 6 years ago

Yes it is.

Especially after you tried configuring an application which has pages-long comments with commented out examples that are misaligned to indentation, and an app takes few minutes to show you an config error.

It is completely impractical - it looks nice - but we aren't writing for YAML for non-technical users, we are writing it for machines and other technical users.

I seriously wonder why TOML isn't popular.

IshKebab• 6 years ago

I think YAML is popular because it has a non-noisy way of including multi-line strings, which are very common in CI /DevOps type things (where YAML is most popular). All of the other features and insanities of YAML don't really matter compared to that.

TOML is good, but it quickly gets confusing when you have object hierarchies / nested tables. I feel like json5 is probably the best option at this point.

MaxBarraclough3• 6 years ago

That stood out to me too. A good article, but a bit unwise to include a dig at the Python language. At the very least we can all agree it's a very successful language. Beyond that, it's the kind of thing people have strong feelings on. If you want to make the case against a major programming language, that should be done in an article of its own.

ziotom78• 6 years ago

I was skeptical too of this sentence, but then I realized that I have never used YAML for files that were more than ~200 lines long. What happens for huge configuration files with lots of sub-trees?

(Comparison with indent-based program programming languages like Python and Haskell is not completely appropriate, because one typically avoids too many nesting levels when writing code. But in a large YAML file you might just need to go deep.)

piaste2• 6 years ago

YAML anchors allow a basic level of reorganization and DRY within your file, not unlike functions in code.

I'm sure that some people will scoff and point to anchors as exactly the kind of complexity a configuration format shouldn't have. But I've personally found them extremely useful and straightforward.

chinigo2• 6 years ago

I'm working with a 12k-line YML file with lots of sub-trees right now, and let me tell you, it's a gigantic pain in the butt.

If it were JSON, I could navigate within a subtree by selecting & searching the text between the `{}` delimiters. But instead, I have to gently scroll downwards, keeping a careful eye on the indentation, to see whether I've moved into the adjacent subtree.

thinkharderdev4• 6 years ago

Agree that invoking Python is probably counter-productive but I think the basic point is correct. I've banged my head against the wall many times trying to figure out which line in my 500 line YAML document is not indented correctly. Bonus point when Helm generates an invalid YAML document but because it is interpreted, the line number spit out by the parser is not a line number in your actual source template.

cageface2• 6 years ago

Significant white space is one of those ideas that’s very appealing in small examples and demos but falls apart in production. I was a big proponent of it myself until I saw it cause some very subtle production bugs.

Now I’m more than happy to endure a little extra typing in order to be able to defer code formatting in complete confidence to prettier.

seanwilson• 6 years ago

I've banged my head against the wall many times trying to figure out which line in my 500 line YAML document is not indented correctly.

How are tags or brackets better here? The "urgh, where's the missing bracket!?" problem has been with us for decades and it's not likely to go away until we move from text editors to structured editors. At least you don't have the problem of misleading looking indentation when the indentation itself is the single source of truth.

Lio• 6 years ago

Saying YAML is too pythonic when really you're talking about a file format used by other languages too is a bit cheesy but I get the criticism.

Semantic white space (originally called "The Offside Rule") predates Python.

I believe it was originally invented by Peter J. Landin[1], and yes the name is a joke about football.

The main problem for me is that it makes copy and paste more difficult. You can't just let the editor reformat for you and you have to be more careful with what you comment out.

Python devs seem to live with that OK though (I'm told) so maybe it's not such a big deal.

1. https://en.wikipedia.org/wiki/Off-side_rule

SiempreViernes2• 6 years ago

The main problem for me is that it makes copy and paste more difficult.

Heh, that makes for a good snark for dismissing the white space complaint, it's all from copy-paste programmers! (Or is more properly black space for all you darkmode lovers)

[...] you have to be more careful with what you comment out.

Not sure if I get what you mean? I think python lacking proper multi-line comments (making things you want to ignore a multi-line string isn't as neat) is an irritant, but I don't get how semantic white space impacts the complexity of commenting?

collyw• 6 years ago

I didn't like Pythons whitespace when I first started using it after years of Perl and Java. Then after maybe 6 months I went back to writing some Perl code. I moved some blocks of code around, then got the dreaded missing brace problem where they didn't match up and I couldn't which brace wasn't matching up. I realized that I never encountered that problem in Python. That was enough to convince me of the benefits.

zaphar2• 6 years ago

Python has the advantage that it is easier to control the level of indentation by breaking things out into functions without compromising the readability of the code.

YAML sort of allows you to do the same using anchors but I find it doesn't help readability as much and sometimes hinders it. I think this might be because in yaml you are describing a data structure and it's far easier to understand that data structure if you can see the whole thing at once.

In python you are describing code and for some reason in my head functions as an abstraction in code make it more not less readable.

xaduha• 6 years ago

It's too easy to produce some abhorrent XML if you don't put much thought into it, straight from tables and objects. And then people are surprised that what they see is revolting, hard to work with and reason about.

Cthulhu_• 6 years ago

IMHO, this argument is shallow, as all 3 formats are text-based. Thus, you can - and should - compress files.

Not only that, but converting data to text, compressing, decompressing and back from text to data is kinda dumb; there's technologies like EXI (https://www.w3.org/standards/xml/exi) that transport XML data in a compact binary format. It may not be readable in transit, but switching the transport or converting the EXI into text-based XML should be trivial if you want to inspect it manually.

But people seem to be reluctant to have mechanical sympathy, even if the data is barely ever read by humans.

MaxBarraclough3• 6 years ago

EXI is 'lossless', right? Trivially convertible back to plain XML? Not much downside then. Ideally the downside that it isn't human-readable shouldn't count for much. Technically that's true of HTTPS too. If it's treated as a communication channel, it shouldn't matter at all.

I suppose it depends how good the tooling is. I get the impression it's not natively supported by web browsers.

kzrdude3• 6 years ago

Is EXI usable in a SOAP service?

zeepzeep• 6 years ago

I don't care about anything regarding XML, except the fact that it can read files and make http requests...

int_19h• 6 years ago

XML had its problems, but I wish the baby didn't get thrown out with the bathwater. We got some of the things back (like comments), but e.g. XML namespaces are an amazingly versatile feature when you need to define extensible schemas, or aggregate data from different sources.

Const-me4• 6 years ago

I would mention date and time. XML has them standardized, see xs:dateTime, xs:time, xs:date. In JSON everyone is inventing their own incompatible representation, can be various ISO string formats, or various numbers (seconds, milliseconds, etc) from some epoch, look at that mess: https://en.wikipedia.org/wiki/Epoch_(computing)#Notable_epoc...

C# has XmlReader and XmlWriter classes in the standard library. Technically they are not event based, but conceptually they’re pretty close to SAX, i.e. they implement streaming forward-only parser, and streaming producer.

tasogare2• 6 years ago

C# also has Linq to Xml and the XDocument et al. classes which are a nice way of reading / writing XML.

I personally like XML. It’s a bit on the verbose side, but that makes is way more easy to figure out the position in the tree than it is in JSON. It is also more easily extensible thanks to attributes and namespaces. It supports comments which is mandatory for any configuration format for obvious reasons. It has powerfull transformation engine as well as schema capabilities. JSON schema keep producing new draft of its spec, yet the result is not on par with XML, and more painful to write.

I’d like for XML to make a come back, not as the behemoth it was once, but for case it makes sense, and configuration files is definitely one of those.

Const-me4• 6 years ago

Linq to Xml and the XDocument et al. classes

Yep, pretty sure the OP meant just them when they placed that green icon in the DOM column and C# row. However, the red cross icon in the table says SAX is missing in C#.

roenxi• 6 years ago

...everyone is inventing their own incompatible representation, can be various ISO string formats...

If only we had some sort of international body to standardise these representations. This and the original articles "JSON has no comments [and this is how you do a comment in JSON]" undermine the complaint on a technical basis.

String with an ISO standard datetime is perfectly reasonable. And quite standard. Use UTC+0 where possible.

tingletech• 6 years ago

CCYY-Www-dd and CCYY-ddd are part of the same ISO "standard" that CCYY-MM-dd are from

Const-me4• 6 years ago

String with an ISO standard datetime is perfectly reasonable

The standard defines more than one format.

Another thing, because it’s not on the spec, people went creative. It’s too late to fix I’m afraid, some widely used APIs already have these seconds in their JSONs.

SamWhited4• 6 years ago

The problem isn't that there is no standard date time format, it's that there is no standard way to represent the standard date time formats (plural) in a JSON document.

dragonwriter4• 6 years ago

XML has them standardized,

XML Schema, a separate and later layer on top does.

In JSON everyone is inventing their own incompatible representation

JSON Schema, like XML Schema, does in fact specify standard time, date, and datetime types and their representation.

pjmlp• 6 years ago

XML remains my favourite configuration format, anything that came later just ends up reinventing the wheel, and badly with half of the tooling capabilities available for XML processing.

kavi87• 6 years ago

Yeah well just try to use XML as your config format and spit JSON out of it for your frontend, if you are using Java you'll quickly realize how obsolete and inelegant is this horror that they dare call JAXB.

jorams• 6 years ago

YAML has 22 ways to write booleans - no less!

I would argue that XML, in practice, has many more. Everything in XML is a string by default, and only an XSD can provide actual type information. Lots of things don't, and those that do often define their own boolean types.

ardy42• 6 years ago

YAML has 22 ways to write booleans - no less!
I would argue that XML, in practice, has many more. Everything in XML is a string by default, and only an XSD can provide actual type information. Lots of things don't, and those that do often define their own boolean types.

I think the problem happens when there are 22 ways to write them in the standard and you only know 16 of them (or 27 of them). You're going to be surprised (by the standard) at some point.

If the standard doesn't define anything, then you at least know the responsibility for the definition is between you and your integrations, so at least you won't be surprised by the standard itself.

As far as standard representations go I think the ideal numbers are 0 and 1: either it should define nothing or pick one unambigous standard representation.

nlte• 6 years ago

Ten years ago, I loved XML mainly because of what I could do with XSLT, and I wrote many websites where the data were XML (static files) and the XSLT scripts acted as the views to output HTML. All that was server-side (ASP or PHP). The equivalent for me today is JSON for then data and the Lodash library to transform them. The environment has switched to JavaScript (Node.js + browser), which was a tremendous improvement. For me, Lodash is the new XSLT - all about data transformation - although they're conceptually unrelated.

fergie• 6 years ago

The author forgot to mention one the biggest functional benefits of XML which is interpolation: the ability to drop XML components into the middle of text, thereby allowing for markup and nesting of elements in text.

<tag>

  This text node contains <magic>unicorns</magic> and <magic>fairy dust</magic>.

</tag>

There is (as far as I know) no sensible way to add the magic above in JSON or YAML.

chrisoverzero• 6 years ago

[…] the ability to drop XML components into the middle of text, thereby allowing for markup and nesting of elements in text.

Is that “allowing” for markup, or is it exactly what an Extensible Markup Language was intended and designed to do? I wouldn’t expect a JavaScript Object Notation to be able to do that ergonomically.

XML’s design as a markup language, and not an object or config language, is clear even in that example. The first character of the contents of `tag` is newline.

kanox8• 6 years ago

This example should that XML is decent as markup but for a configuration or data interchange format this kind of syntax is undesirable.

You have text with interspersed elements with the same name, how can this be represented in python other than a DOM?

SamWhited4• 6 years ago

One of the other benefits of XML the author mentions solves this: if you don't want it, disallow it in your schema.

skywhopper• 6 years ago

Right, this, and the lack of typing in XML, are the fundamental differences and the sources of many of the differences in performance, complexity, etc. The question to ask is, what’s the right format for your software’s needs? More often than not, programmatic data structures map more directly to JSON/YAML’s simple and typed fundamentals. That easy mapping plus far better performance are the reasons why people avoid XML these days.

The author of this piece gets it fundamentally wrong in saying that “Enterprise” stink was the reason people abandoned XML. For enterprises that need those features, XML is still a fine format to use. But for most purposes, those features aren’t necessary, and the massive overhead that comes along with them makes it a bad tradeoff.

hyperpallium2• 6 years ago

The old saying is XML is a markup language, JSON is an object notation.

chrismorgan2• 6 years ago

Here’s the thing: each element needs to know what type its children are. Does it contain only text nodes, or can it contain elements? You can’t just put markup in a place that’s only equipped to handle text and expect it to do anything but explode in interesting ways. (“Interesting” as in “may you live in interesting times”.)

The same can be applied to other formats like JSON and YAML: if it contains only text, use a string, otherwise use an array of strings or objects of some defined structure. Here’s one possible representation:

  {
    "tag": [
      "This text node contains ",
      {"magic": "unicorns"},
      " and ",
      {"magic": "fairy dust"},
      "."
    ]
  }

So I don’t consider what you describe to be a benefit of XML at all, except inasmuch as its syntax can be subjectively a little prettier.

maxerickson• 6 years ago

Add nesting to the xml and the array of array of arrays starts to look a lot worse.

jerome-jh4• 6 years ago

A good example of JSON stupidity. Square brackets introduce an array, whose elements should be of consistent types. But actually this is not enforced. It only enforces the ordering of elements, which adds quite a bit of information! So this is a list (called an array).

Curly brackets introduce an "object", whose fields are considered unordered. So you are not supposed to have multiple "tag". What happens if this is the case ... unspecified!

JSON is probably worse than XML here.

larkery• 6 years ago

This is like a strictly worse kind of s-expression. C.f.

    (tag "This text node contains" (magic "unicorns") " and " (magic "fairy dust"))

jimbokun• 6 years ago

It is always sad and funny watching the world churn through inferior plain-text-structured-data solutions, while studiously ignoring s-expressions.

jayd16• 6 years ago

Json is not about markup and shouldn't inject surprise objects into the middle of strings. You wouldn't use json for this.

ajxs• 6 years ago

Here's a fantastic article about the various misuses of XML that I greatly enjoyed: https://www.devever.net/~hl/xml

fetbaffe2• 6 years ago

In the old days many things that were XML when it should have been JSON, today it is the opposite, many things that are JSON when it should be XML.

ivolimmen• 6 years ago

I so agree with this. I have been complainging in different companies that using REST is not practical between application (at least use JSON-API then). REST is usefull for frontend <-> backend communication. SOAP was horrible simetimes but it worked. If you change the contract the clients simply do not compile. This is not the case when you change a service using REST (yeah there are some fixes for that but they are not common and widely used).

dragonwriter4• 6 years ago

I have been complainging in different companies that using REST is not practical between application (at least use JSON-API then).

JSON-API is REST, though not all REST is JSON-API.

REST is usefull for frontend <-> backend communication.

Actual REST is least useful for communication between a single frontend and backend controlled by the same party.

SOAP was horrible simetimes but it worked.

More accurately, SOAP was horrible and it sometimes worked.

If you change the contract the clients simply do not compile.

I’m not sure how that would be a good thing, even if it were true, which it isn't unless you are fetching the WSDL and then recompiling clients each time you try to access the server.

This is not the case when you change a service using REST

Which is a good thing, HATOEAS plus resources described with media types and using content-type and accept headers properly, all of which is part of REST over HTTP, give you plenty of opportunity for clean error for a representation change you can't handle without requiring you to try compiling at each access.

theSIRius• 6 years ago

Actual REST is least useful for communication between a single frontend and backend controlled by the same party.

I know I'm going off-topic but what is best way to communicate between frontend and backend except for REST? I know about GraphQL even though that is not widely adopted yet and in a lot of cases never will be.

indymike3• 6 years ago

I've been involved in XML standards making for about seven years, and one of the things I'm most proud of is encouraging the standards body to publish a JSON standard.

XML is extremely flexible, and delivered on being extensible. That is why it has fallen out of favor. XML is pretty unforgiving in terms of what is valid (this is all for good reason) and the extensibility brings with it a lot of complexity. XML is:

- Complex and requires tools with more functionality than, say JSON. Yes other formats do less. That is why formats like JSON are supported completely in most languages and XML and it's dependent standards. For example SOAP is not fully supported by Python.

- Brittle. It is easy to introduce errors in XML that cause most XML parsers to fail. Bad characters, bad or missing escapes, unbalanced tags all introduce points of failure.

- XML is the foundation of even more brittle and complex standards like SOAP and XHTML. Yes, SOAP is really cool. WSDLs when supported are magic. But... when something isn't right, it's very difficult to fix or get working. Meanwhile, a mediocre developer can integrate with a web api that delivers JSON in an hour or two.

While XML can do the job, JSON, YAML and TOML all are a lot smaller in terms of tooling and cognitive overhead, and most importantly, writing a library to handle them is much less daunting than supporting XML and all of it's child standards.

All that said, for more complex data interchange cases, and internal file formats, XML still is hard to beat. So much work is done for you.

wsc981• 6 years ago

Whenever someone mentions SOAP I’m always reminded about this funny article “S Stands For Simple” [0].

———

[0] http://harmful.cat-v.org/software/xml/soap/simple

dfox• 6 years ago

It is "Simple" in contrast to things like CORBA and DCOM. Which it accomplishes by not solving the main problem that these distributed object protocols try to solve at all and instead being overly complex RPC protocol with completely unnecessary "object orientation".

On a similar note, the L in LDAP stands for lightweight, which was accomplished by removing advanced authentication features from X.500 DAP and then gradually reimplementing the same functionality with significantly more complex "simple" "framework" that in theory allows usage of any conceivable authentication mechanism, or something like that.

quercusa2• 6 years ago

I lived much of that story, and it's only the slightest exaggeration.

janci• 6 years ago

I see the main problem of XML in it's model not matching the data model you usually work with. In code you work with arrays or structs/objects that have members. Members can be other structs or arrays or primitive types. Arrays can be composed of structs, arrays or primitive types (sometimes heterogenous, but not very often).

This does not map to DOM at all, in any direction. In XML DOM you have structured nodes intermixed with text, order of nodes is significant (or may not be) and you have attributes. On top you have namespaces.

Indeed, you can restrict to subset of XML and find a mapping that works, guaranteed you will get a very verbose XML that is hard to work with.

throwaway8943452• 6 years ago

I would generally avoid XML, but the alternatives aren't perfect either, which is to say I sympathize to a degree with people who are familiar with XML and dissatisfied with the shortcomings of the alternatives. YAML and TOML are both pretty complex in their own rights--really what I want is JSON with comments and multiline strings for the "human reader/writer" use case. Never the less, I've never been so pained by these deficiencies that it made sense to use XML.

jle17• 6 years ago

really what I want is JSON with comments and multiline strings for the "human reader/writer" use case

Sounds like you want json5: https://json5.org/

It also has the "Arrays may have a single trailing comma." feature which is a pain to not have in json.

afiori4• 6 years ago

encouraging the standards body to publish a JSON standard.

Were you successful?

indymike3• 6 years ago

Yes.

gnufx• 6 years ago

I don't see a reference to Siméon and Wadler: "So, the essence of XML is this: the problem it solves is not hard, and it does not solve the problem well." http://homepages.inf.ed.ac.uk/wadler/papers/xml-essence/xml-...

60secz• 6 years ago

XML did many things right and a few big things wrong. It's a tragedy there isn't a single imple, standardized way to define an array through xml as there is through json. The lack of schema validation in json is equally baffling.

A hack for JSON comments

{ "//": "Comment1", ... "//": "Comment2" }

sharpercoder• 6 years ago

Json-schema.org

dragonwriter4• 6 years ago

The lack of schema validation in json is equally baffling.

XML schema languages, like those for JSON—of which there are many, again as is the case for JSON—appeared after the format had wide adoption, and are separate from the base standard.

I'm not sure what distinction you are trying to draw here, because I don't really see one.

Mikhail_Edoshin3• 6 years ago

XML is in a totally different category than JSON: it's a tool to craft formal languages. A data format or a config is also a language of sorts, so it can do that as well, but it's real purpose are languages in general. XML languages are what is left of a language if you omit parsing and start with AST.

Configs and data formats normally don't require all the benefits of a language (e.g. composability), but other things do, e.g. UI systems.

patrec3• 6 years ago

This is terrible advice. Exposing any public endpoint that talks XML is a security disaster in waiting and the fact that the author makes no mention of a long history of major security vulnerabilities in popular XML packages does no credit to him.

The "composable" without "confusion about semantics" is a joke, too: in practice it's not even possible to know what whitespace means. Is it semantically meaningful or just there for pretty printing? Also, say you see a commented out region in an XML file. How do you uncomment it? If XML where less braindamaged both problems would be trivial and purely mechanical. In practice you'll generally have to guess.

All in all XML is an extremely mediocre design: it is substantially worse than prior art in countless respects and tremendously complicated without achieving much of anything.

jerome-jh4• 6 years ago

Indeed. Although XML has a grammar its complexity led to countless flawed implementations. The original sin of XML is that it has been over-hyped from the beginning. Is it the cause or the consequence of it being over-engineered, I would not say. Anyway following standards, XLST, XPath, saw poor adoption. I am not fond of JSON and YAML either, but since they are much less ambitious, I am somewhat more forgiving with them. None of them is good at being a config file format. None one them is good at data serialization, that should be binary and TLV encoded.

WesolyKubeczek• 6 years ago

it is substantially worse than prior art in countless respects

The prior art being? ASN.1 libraries don't have a stellar security track record either.

Flexible data interchange is _hard_.

patrec3• 6 years ago

XML is basically just a vastly inferior form of sexps, just as ASN.1 at least for the purposes of encoding cert info in TLS is vastly inferior to csexps -- the latter is canonical by design, efficient and trivial to write a robust parser for.

sergeykish8• 6 years ago

I had whitespace removed by XSLT moment. Why would a format designed for application consumption have insignificant whitespace?

Also HTML whitespace handling makes all sort of bugs in contenteditable. Chrome inserts   instead of space, finally found solution

    * { white-space: pre-wrap }

Still it removes first newline in <pre>, at least that is solved in XHTML.

I would love to see how XSLT can be implemented on s-expressions. Attributes are not required.

tester34• 6 years ago

XML is great for interfaces, ain't it?

nickbauman• 6 years ago

I think the problem with XML is that people tried to make it into something as comprehensive as CORBA (this is SOAP) so that all the decisions about the communication was done in-band. JSON requires everyone to make out-of-band decisions about what the JSON should look like: people talking to each other, imagine that! The former seems quaintly antisocial nowadays.

piercebot• 6 years ago

Surprised that nobody has brought up size over the wire yet. XML, by virtue of the fact that it has both opening and closing tags, tends to be more verbose than other standards. With larger requests, this can translate to hundreds of extra kilobytes transferred. Even for smaller requests, people in bandwidth-constrained environments will probably notice performance improvements when using a more concise standard over XML.

hobs2• 6 years ago

Especially for repetitive files, I did some testing for an ETL process that was effectively "here's multiple CSVs in an XML file" - break that into multiple csvs (which matched the customer expectations anyway) and got 30x savings in size.

MereInterest2• 6 years ago

How much of this is effectively making a custom compression scheme? Would you get better compression by using the gzip compression of HTTP?

theandrewbailey• 6 years ago

XML's verbosity is solved with compression. Your webserver is configured to compress text-based formats, right?

rjkennedy982• 6 years ago

I think his notes about performance miss the full picture. Obviously gzipped XML and JSON aren't going to cause network slowness, but I've been on big projects that use XML as a data interchange format and XML marshalling and unmarshalling with validation was a bottleneck in the system. We used JAXB and loading the hundreds and hundreds of schemas in JBoss was so expensive that they had to bump up the memory to 32 gb on our personal computers. In the end XML schema validation had to be turned off (removing the whole point of it). Also, this was Java which is compiled and obviously very fast. Can you imagine the same thing in PHP or Python? It would be impossible. I'd be curious if the author ever did a large XML project in a dynamic programming language. I seriously suspect not and these languages (PHP, Node, Python) are the most popular and productive languages in the world.

XML is good at some specific things, but as a general use data-interchange format it is dead as a doornail and there is no good reason to bring it back.

fetbaffe2• 6 years ago

Most common XML parsers in PHP uses libxml which is written in C, so it should be relatively fast, however I haven't used it for super large XML-files and not for every request. It would be interesting to test.

textman• 6 years ago

XML was designed for documents, not data. By document I mean large amounts of text. It came out of the publishing industry. In that context, the tags end up comprising an insignificant portion of the total document, so is not overly verbose.

In Defense of XML

Related Stories