The XML spec is 25 years old today

https://www.w3.org/TR/1998/REC-xml-19980210

by klez • 3 years ago

134 193 3 years ago

XML vs JSON is such a fascinating topic, a clash of different approaches, academic vs practical might be one way of describing it but I'm sure there's many others. Are there any good books or essays on it?

And having been there in the 90s seeing XML and thinking "they've tried to do everything and got into a big mess" (XSLT anyone?) its so satisfying to see something simple and flawed displace it.

magicalhippo• 3 years ago

In a sense though the circle has been closed. With the latest JSON Schema spec, we were able to do a 1:1 mapping between our XSD's and JSON Schema, so now we can accept JSON as well as XMLs.

For small messages though I agree JSON is smoother.

didntreadarticl6• 3 years ago

Whenever I hear people talk about JSON schema spec it reminds me of XML in the 90s.

quickthrower2• 3 years ago

XML has a unique spot in the intersection of payload and human crafted file.

JSON is something I see as payload-only really. And YAML etc are human-craft only.

One thing I would change in XML is that closing a tag should not need to specify the name. <> should close any tag. It would make editing a bit easier and save a byte or two.

didntreadarticl6• 3 years ago

Its true, I can bang together an XML doc by hand with less worry than if I type out some JSON

bazoom427• 3 years ago

SGML allowed you to minimize a closing tag. And HTML supported this in theory, although mainstream browsers never supported it.

XML went for a syntax with fewer options because the complexity of SGML was considered a hinderance to its adoption.

riggsdk• 3 years ago

My web-technology professor at my uni (around 2007) was a big fan of XML and all the surrounding technologies. He saw it as this beautiful interconnected system where the web being pure XML would allow all data in the world to be queried like a database and transformed into any format you wanted.

The web sure didn't go that way...

didntreadarticl6• 3 years ago

Thats "XML - the religion"

kzrdude3• 3 years ago

Basically, he was in love with serialization and connecting services together, that part is cool. It's just a lot more diverse in serialization now than before. And a lot of these projects compete in reducing overhead - they are orders of magnitude more efficient than XML.

timeon2• 3 years ago

I think it was called semantic web or Web 3.0.

daitangio• 3 years ago

In my huble experience:

- XML parsers were quite bad in the old days (2000), even in java land.

- Parser speed is a functin of the input size. And XML inputs are quite big, too big in my humble opinion.

- you have tag attribute and tag value, and so people get confused on how to use them for simple scenario.

- & must be escaped but no one does it. So when 'AT&T' ended up in a stream created by "hand" by a COBOL procedure, the XML got suddenly broken

- < and > musty be escaped, and so your SQL queries must be escaped

JSON is simply an associative hierarchical maps. A Lot of 1995 PHP code used the same data structure and just works. No attribute, only hierachy recursive structure.

Recusion always wins.

adrianmsmith• 3 years ago

That's all true, I'd just add that in JSON you have to escape " in strings just like you have to escape & < and > in XML, so there is still the potential for e.g. COBOL software to produce invalid files.

bambax3• 3 years ago

Parser speed is a functin of the input size. And XML inputs are quite big, too big

You don't have to parse the complete document to do something about it. You can use a streaming parser, and you can execute XSLT on the fly.

Gibbon12• 3 years ago

Around the time xml was released I asked a (smarter than me) friend what the hubbub was. And he got a pained look on his face and said. People think it'll solve a problem and it won't.

His reasoning was the problem isn't standard file formats. It's turning those into something a program can operate on as data[1]. And XML does not do that for you. No you have suck it in and then manually transform it into 'data'. So it just moves the problem from one place to another place. Which is a trash solution usually.

Far as I know JSON broken as it is does do that. At least with dynamic/ish languages.

[1] gross stuff people used to do like copy data structures in memory directly to a file actually solves that problem while being a maintenance nightmare.

cmrdporcupine2• 3 years ago

In many respects the failure of XML to live up to the grandiose promises made in the 90s was due to the lack of decent tools around XML. Decent structure-aware, refactoring, editors didn't exist then, and barely do now. XML cries out for something like paredit, but dumbed down and tag aware. The author shouldn't even have to see the textual representation of the tags.

But, primarily, the thing that made JSON win out is that it reflects the existing semantic/data-structure model of an actual programming language (and approximates that of many others.)

XML (and XSchema, oh god), has this mixture of associative and sequentially ordered data, and two separate kinds of associative/nesting models (elements vs attributes). It wants to eat the world with its new semantic model, but it doesn't offer any particular semantic advantages and doesn't match what programming languages (or relational databases for that matter) work with.

And then on top of that the syntax is just ugly to read.

Still there was a moment in the 90s where I found XML kind of exciting and interesting. I don't really recall why now? Seemed everybody did.

rprospero• 3 years ago

I also remember being excited about XML in the 90s and I can recall why. At the time, every application I used had its own bespoke file format and needed a custom parser to read its output. This especially applied to several applications storing hierarchical information in CSV files with their own way of delimiting CSV data as content in a CSV cell. The switch to XML meant that I'd have to figure out the tag soup that the developer had chosen for their XML data, but that was so much more comfortable than figuring out the goulash of ASCII escape tags in the old format.

XML is like the pager. It might seem clunky today, but it's better than having to tell the baby-sitter the phone number for the restaurant and theatre you'd be visiting that night.

raverbashing2• 3 years ago

XML seems like the apex of "design by committee" (and I mean, didn't we have some thick books about it? What for?)

Nobody cares about 99% of that

pulse7• 3 years ago

I am happy we are not using SGML, which is a base for XML and is MUCH MORE complex and is also a standard...

bazoom427• 3 years ago

XML itself is a pretty focused spec. XML schemas and the whole SOAP stack is where it went off the rails.

Devasta3• 3 years ago

XSLT is great, the problem is that the browsers stopped at XSLT 1.

XSLT 3 is a different beast, and XSLT 4 is being worked on at the moment.

bawolff4• 3 years ago

I think the true lesson of xslt, is xml makes a terrible syntax for a programming language. The language itself minus the syntax is fine, although i think many people prefer a more imperative approach.

doctor_eval2• 3 years ago

I wrote many thousands of lines of XSLT back in the day, converting XML into XSL:FO and pumping it through FOP. It worked remarkably well. It was my second experience with a declarative language (SQL being the first).

There were some great resources - libraries of code fragments, similar in spirit to tailwind ui - and people did crazy things with them.

We eventually abandoned it, after many years. It was indeed terribly difficult to work out how a document got transformed, and when returning to a transform after a long time working in other languages, you could easily spend a day just trying to work out how some trivial thing worked.

I remember it fondly, but I wouldn’t do it again.

_glass2• 3 years ago

Sometimes I just don't really see it as a competition. For my research work often XML is a markup, which JSON is not. If I want to qualitative code a text, this is perfectly doable by hand:

  <note>
    <paragraph>This is a <code vibe="positive">remarkable</code> text.
    </paragraph>
  </note>

If you do it in JSON, it is not really readable anymore, and I would have to write GUI for the input, e.g.,

  { "note": 
    { "paragraph": 
      [ "This is a ",
        { "code": [ { "vibe": "positive" }, "remarkable" ] },
        "text." 
      ]
    }
  }

nordsieck• 3 years ago

IMO, you made the JSON unreadable with all the extra whitespace that you didn't add to the XML.

    {"note": 
      {"paragraph": 
        ["This is a ", {"code": [{"vibe":"positive"}, "remarkable"]}, "text."]
      }
    }

Also, in this particular case I think the child/property dichotomy works in the favor of XML, but typically I find it to be more of a liability than an asset.

bambax3• 3 years ago

XSLT anyone?

I used to love XSLT (I would probably still love it if I still used it, but the projects have mostly disappeared). Pattern matching is akin to event driven programming: just deal with what you have, if and when you have it. It was very clean and never broke.

throw0101c• 3 years ago

XML vs JSON

Fun fact: Tim Bray was the/an editor on both specs:

* https://www.rfc-editor.org/rfc/rfc8259

* https://www.w3.org/TR/xml/

I wonder if he'll commemorate/note the anniversary in some way:

* https://www.tbray.org/ongoing/

college_physics5• 3 years ago

Are there any good books or essays on it?

That would be a really good resource. I mean, anybody who has worked remotely with web technologies has an opinion about this topic but it would be interesting to compile an objective knowledge base and interpretation of why things evolved this way.

innocentoldguy• 3 years ago

I despise XML, not because it was a bad idea, but because of all the bad ways I've seen people use it.

For example, I used to work at an ed-tech company that bought bubble test questions from Pearson, who provided their data in HUGE XML documents. If I remember correctly, they would do things like splitting the sentences of the test questions up so half the question was a tag attribute and the other half was an element. So, instead of just parsing questions, we'd have to parse them and then stitch them back together to make complete sentences. They did that with the answers too, I believe. So weird.

The JSON format makes it harder to abuse data like my example above and it is a lot easier to parse, so I'd reach for it before XML any day.

Thiez• 3 years ago

Sounds like something that XSLT could have solved. When in doubt use more XML.

kzrdude3• 3 years ago

I would want experiment with root structure of XML.

Make it possible to have multiple root elements. One problem is that however big an XML document is, it has to be closed at the end, so it ends up as one atomic element, you can't partially parse an XML document correctly.

planede9• 3 years ago

What do you mean by "partially parse"?

didntreadarticl6• 3 years ago

If you imagine a log file that gets lines added to it now and then, its actually quite fiddly to do that with XML because of the closing outer tag

oevi• 3 years ago

XSLT 3.0 allows streaming for large XML files. See e.g. https://www.saxonica.com/html/documentation10/sourcedocs/str...

rhdunn• 3 years ago

Isn't that what SAX (event based) and reader (pull based) XML parsing is for? Those allow you to incrementally/partially parse XML.

phire• 3 years ago

Well... There is nothing stopping you partially parsing an XML documents. What you can't do is validated it. Which is the same for any other file format. You be sure the file is fully valid without fully parsing the whole file.

However, the only order you can partially parse an XML file is linear order, which clashes badly with the fact that it's a tree based format. Depending on how tree-like your schema is, this might be a massive hinderance.

This flaw isn't unique to XML. All text encoded file formats share this characteristic, and can only be parsed linearly. If you move across to the world of binary file formats, it's extremely common for them to have indexes of offsets so a parser can navigate a tree-like structure in tree order without having to fully parse it, along with other types of non-linear data structures.

HelloNurse3• 3 years ago

All text encoded file formats share this characteristic, and can only be parsed linearly. If you move across to the world of binary file formats, it's extremely common for them to have indexes of offsets so a parser can navigate a tree-like structure in tree order without having to fully parse it, along with other types of non-linear data structures.

You don't always have the luxury of randomly accessing a file (obvious example: shell pipelines with a producer and consumer exchanging lots of temporary data), so taking advantage of indexing might require saving a temporary file and stalling processing until the file is ready.

Personally, I'm used to parsing large XML files with event-based APIs and throwing away data aggressively, keeping in memory only one unfinished element of interest, the stack of its ancestors, and my collected data (instead of a DOM for the whole document).

bawolff4• 3 years ago

Its super common for xml parsers to support partial documents.

mdaniel• 3 years ago

and, AIUI, XMPP is based entirely around that very idea so it's not even a niche concept (well, aside from XMPP being arguably a niche protocol)

orra2• 3 years ago

XML 1.0 was clearly a huge success. What I'm curious about is the seeming failure of XML 1.1. What happened there? Is it just like IPv6, in that there are too many non-upgradable legacy systems?

planede9• 3 years ago

It's like YAML 1.2

dfox2• 3 years ago

XML 1.1 did not "fail". It is from the start designed as slight relaxation of XML syntax rules for applications that can benefit from it, which essentially means applications running platforms where the usual encoding is not derived from ascii (eg. EBCDIC). It was never meant to be widely used.

orra2• 3 years ago

Of course XML 1.1 was meant to be widely used. The design goals[1] for XML 1.1 are the same as for XML 1.0, including being "straightforwardly usable", "support[ing] a wide variety of applications", and being "easy to create".

The relaxation of names in XML 1.1 is good for internationalisation[2], by not limiting names to Unicode 2.0 characters. Allowing NEL characters, as used on IBM mainframes, was not the sole motivation for XML 1.1.

[1] https://www.w3.org/TR/2006/REC-xml11-20060816/#sec-origin-go... [2] https://www.w3.org/TR/2006/REC-xml11-20060816/#sec-xml11

college_physics5• 3 years ago

Its a quarter of a century already... Enough of an empirical observation window to conclude that the way different entities choose to implement or adopt online information exchange is a tortured, highly inefficient process that is ultimately just a collection of historical accidents and technological aspects or merit are secondary

On the face of it is strange that there is so much friction [0]. I will just enumerate a number of bizarre outcomes that I am aware of and I am sure there are others:

* the xml web still goes strong both as-is and in large number of popular or niche-but-important domains (from html, svg to xbrl, sdmx, ...)

* the semantic web (rdf, owl etc.) and making some sense of all the data exchange with metadata has gone nowhere

* the API web (json) is reinventing the wheel for validation, annotation (schema etc)

[0] xkcd jokes aside, people do have, in principle, strong incentives to adopt common languages and standards.

Devasta3• 3 years ago

And here's to the next 25!

curiousgal• 3 years ago

I highly doubt computers will still exist in 1.551121e+25 years!

pantulis• 3 years ago

And coffee was spilled! Well played!

didntreadarticl6• 3 years ago

[factorial joke]

lp4vn5• 3 years ago

XML was supposed to be a silver bullet of the myriad of ad-hoc formats for structuring documents that existed in the wild pre 2000(I got this information contested here, but that's what an old professor told me).

In the XML "stack" you can find the XSL to ensure the validity of an XML document, the XPath to be a querying language, the XSLT to be a language for writing transformations, etc. I think one must at least respect XML stack vision of coherence and completeness.

Then you have things like JSON where the whole wheel is being reinvented, in my opinion, for no clear purpose. You have for instance hundreds of libraries for creating a JSON schema and everything looks like a waste of time overall.

Before anyone asks me, yes I have used JSON and YAML and I never could really understand why people keep recreating basic stuff like markup languages instead of sticking to whatever is available. And yes, verbosity because of the XML tag system always sounded a pretty silly argument to me.

bawolff4• 3 years ago

Because worse is better (which i mean in the unix sense of the phrase).

On the specific subject of schema, i think most of that is most users have no need or interest in a schema.

college_physics5• 3 years ago

most users have no need or interest in a schema

that is a strange statement. users must reach out for a schema practically as soon as some data is exchanged

the only users that don't need a schema are the ones operating within a controlled environment where everything has been somehow agreed or imposed (e.g. within a walled garden or when you control both sides of the exchange)

missing, incomplete, low-information schemas about what data is supposed to mean and how it can be used correctly etc is a major, major wast of time. in data science circles the joke is that 90% of the work is "data cleaning". what do you think data cleaning is about?

lp4vn5• 3 years ago

On the specific subject of schema, i think most of that is most users have no need or interest in a schema.

You have no idea, my friend. I came many times across the situation where people needed to have a JSON conformant with a schema and they just resorted to the ad-hoc badly written json-schema library du jour.

bambax3• 3 years ago

Completely agree! But I think you meant "XSD" here:

you can find the XSL to ensure the validity of an XML document

lp4vn5• 3 years ago

Thanks!

valenterry• 3 years ago

JSON is not a reinvention. It's two different things. XML is a markup language. It's for markup, so, do you need markup? Then don't use JSON. You don't need markup? Then don't use XML.

GoblinSlayer3• 3 years ago

Markup is syntax that gives structure to text. Ironically there's a port of HTML to JSON syntax. Guess who did it.

college_physics5• 3 years ago

for no clear purpose

performance may have been one driver, though for the "API web" which is the main use case there are better performing options than json

everything looks like a waste of time overall

maybe all that matters in the end is to be gainfully employed. having multiple formats and standards helps with that. when the Euro got introduced in Europe thousands of FX traders busy trading national currencies had to switch to trading real estate.

lp4vn5• 3 years ago

maybe all that matters in the end is to be gainfully employed. having multiple formats and standards helps with that. when the Euro got introduced in Europe thousands of FX traders busy trading national currencies had to switch to trading real estate.

Agreed.

dtech• 3 years ago

The reason is pretty clear, XML and its ecosystem was (is) enormously complex. When naively - but correctly - parsing introduces serious security vulnerabilities [1] you kinda have problems as a format. XML is just incredibly hard to get correct, understand, and be performant. JSON from the get-go was very easy to understand and start with so it gained the upper hand.

[1] https://owasp.org/www-community/vulnerabilities/XML_External...

butlerm7• 3 years ago

I wouldn't consider that a parsing vulnerability - XML is relatively easy to parse - but rather an XML processing vulnerability, as in you must disable certain features (like arbitrary filesystem access) when processing an untrusted document.

With hindsight, an XML processing library should require that you turn features like that on, rather than require that you turn them off. Opt in to dangerous features rather than opt out of them in other words.

tannhaeuser2• 3 years ago

that's what an old professor told me

Well then your prof has no idea what (s)he's talking about I'm afraid. XML is just what the spec says on page 1 sentence 1 [1]:

The Extensible Markup Language (XML) is a subset of SGML that is completely described in this document. Its goal is to enable generic SGML to be served, received, and processed on the Web in the way that is now possible with HTML.

Where "generic SGML" refers to the property of XML being always fully tagged and not needing markup declarations, but still able to make use of those for validation and limited forms of entity expansion. Full SGML OTOH uses markup declarations for tag inference/omission (as in HTML), attribute short form canonicalization (also as in HTML), Wiki syntaxes (as in markdown), stylesheets (as in CSS), and other features such as type safe/injection free templating, processing pipelines/document "views" and as general extension mechanism.

[1] https://www.w3.org/TR/REC-xml/

EdiX3• 3 years ago

XML was a bad fit for almost everything people did with it. The natural data model of XML is alien to all programming languages people use: it is alien to scripting languages because you want hashmaps and lists and it is alien to static languages because you want structs and lists. So you usually ended up having to have a two stage parser, with the first stage turning XML into some kind of DOM and then your handrolled DOM-to-internal-representation thing. That's also most of the justification for XML's ecosystem:

- You need XSD to make sure that your DOM-to-internal-representation code doesn't have to deal with garbage

- You need XPath to query XML directly if you don't want to convert it to something usable

- You need XSLT as a scripting language where the DOM isn't alien and awkward to manipulate

This is why there is no xslt equivalent (although you could argue that jq is that) and the various json schema solutions don't see much adoption, despite json being almost as old as XML at this point.

lp4vn5• 3 years ago

Honestly I don't agree with you, the abstract concept behind XML is just to provide a metalanguage for the representation of hierarchical data, and that's exactly what JSON and YAML do. They are completely interchangeable. XML is the general case for hierarchical data and provides tools for this general case. JSON got closer to the representation of list and hashmaps, but I see this as a mere convenience.

This is why there is no xslt equivalent (although you could argue that jq is that) and the various json schema solutions don't see much adoption, despite json being almost as old as XML at this point.

I think it's incorrect to part from the assumption that nobody's gonna need a schema language, a query language, a mapping language or whatever. For the json schema this is blatantly untrue and you see many people creating their own version of a json schema because it's really needed. With XSLT you have a language for processing XML that's written in XML itself, what I find pretty nice. We're not gonna see a YAML processing language written in YAML any time soon though.

butlerm7• 3 years ago

I fail to see why any typical use of XML _needs_ XSD, XPath, or XSLT. You certainly don't need XSD validation. It is trivial with stream processing to detect, skip over, and report on elements that shouldn't be there and every general purpose XML alternative has exactly the same issue, if not a worse one through use of eval style import of objects of questionable provenance.

Then you have no idea what your object structure now imported natively in your language contains, and you would have to post process it for vulnerabilities using introspection to find out.

In my view there just is no substitute for careful analysis of data submitted by any third party, and parsing an object into a tree structure and validating it after the fact is a questionable practice with untrusted data from anywhere. It may consume an arbitrary amount of resources before you have any idea whether the data in question bears even a passing resemblance to what it is supposed to be. If you have untrusted data you really shouldn't be using an unrestricted DOM constructing parser or eval style processing at all.

otabdeveloper42• 3 years ago

JSON is not a markup language, it is a serialization format.

kaba03• 3 years ago

A bad one at that. Not the worst (which would be yaml), but certainly not great.

NoboruWataya2• 3 years ago

I actually didn't realise the spec was so young. I started using the internet around the same time as it was finalised. It's something you just assume has always been there.

XML has its issues and personally I am always happy when I am dealing with an API or data source and see JSON rather than XML. However, there are a lot of cool things built on top of it. RSS is an obvious one that pretty much everyone here will appreciate. GPX is another really useful one. A lot of financial reporting is now also being standardised using XML. Even derivatives trading: https://www.fpml.org/

martinclayton• 3 years ago

I had a similar feeling. I think it's because XML relatives HTML and SGML (GML anyone?) go so much further back.

butlerm7• 3 years ago

HTML was invented in 1993, which is not that much further back. GML was invented in 1969, and SGML in 1986. I guess you could say that HTML resembles GML quite a bit with the addition of angle brackets and urls. XML on the other hand is basically SGML with a number of optional features left out.

leashless• 3 years ago

I worked on XML-EDI "back in the day". When I came to the blockchain (Ethereum team 2014) I had high hopes that it might provide the architectural solution to the problems with XML in terms of workflow, API endpoints, and the (infamous) failure of WSDL ("web services description language"). I wrote it up in the Ethereum launch piece here: https://medium.com/humanizing-the-singularity/by-the-end-of-...

=== quote begins ===

Of course, there are attempts to clarify this mess — to introduce standards and code reusability to help streamline these operations and make business interoperability a fact. You can choose from EDI, XMI-EDI, JSON, SOAP, XML-RPC, JSON-RPC, WSDL and half a dozen more standards to assist your integration processes.

Needless to say, the reason there are so many standards is because none of them work properly.

Finally, there is the problem of scaling collaboration. Say that two of us have paid the upfront costs of collaboration and have achieved seamless technical harmony, and now a third partner joins our union. And now a fourth, and a fifth. By five partners, we have 13 connections to debug. Six, seven… by ten the number is 45. The cost of collaboration keeps going up for each new partner as they join our network, and the result is small pools of collaboration which just will not grow.

Remember, this isn’t just an abstract problem — this is banking, this is finance, medicine, electrical grids, food supplies, and the government.

Our computers are a mess.

=== quote ends ===

My company uses XML in a sort of "semantic web" paradigm to describe products, and links to these documents from smart contracts to provide accurate machine-readable descriptions of things which are for sale. It works reasonably well, and maybe it'll turn into something a lot more powerful as operations expand. So we do walk the talk to some degree.

mrgriscom• 3 years ago

Always a classic: http://harmful.cat-v.org/software/xml/s-exp_vs_XML

NoboruWataya2• 3 years ago

They are not identical. The aspects you are willing to ignore are more important than the aspects you are willing to accept. Robbery is not just another way of making a living, rape is not just another way of satisfying basic human needs, torture is not just another way of interrogation. And XML is not just another way of writing S-exps. There are some things in life that you do not do if you want to be a moral being and feel proud of what you have accomplished.

Wow.

goto114• 3 years ago

If there is an argument beneath the hyperbole and i-am-very-smart posturing, it seem very weak. He end up advocating a binary format (like TCP/IP) instead of a text-based like XML because it will be faster to parse. And because a binary format is harder to change, people will be sure to get it right the first time.

Yeah right.

bazoom427• 3 years ago

XML shines at the intersection of structured documents and structured data. For pure data you can use JSON or s-expresssions but this does not work very well with more document-orinted data. There is a reason frontend frameworks use HTML templates or JSX even though js supports JSON natively.

crabbone5• 3 years ago

Why do people advocate for JSON so much? It's simple, I can see that, but it also got most of its stuff wrong. If I had a liberty to choose a format in which I want to receive data, JSON would've been very much down on the list.

These are just the problems that actually bit me in my professional use of JSON, not some theoretical possibilities:

* No bounds on how big integers may be, subsequently, an ambiguity about what to do if the number is bigger than what would be considered a typical integer size for the platform: should it be interpreted as a float or as an integer? In my case, I had ids coming from a database that used 128-bit to represent them and the client side silently converted them to floats, but, on the way back, converted them to integers again, which resulted in rare collisions and lost data.

* No defined semantics for repeated keys in hash-tables. I used a library that tried to stream JSON in SAS-like way, but the authors didn't realize that duplicate keys are a possibility. Again, this caused subtle and hard to identify data integrity issues.

* Unnecessary null value. For some languages null and false are the same thing (and I prefer it that way too), but converting data back and forth between JSON and language-native data would sometimes produce inconsistent results, and, again, caused a once-in-a-year case of data integrity issue.

* Mandatory Unicode string encoding. Caused problems with DICOMs from East Asia. DICOMs have some special encodings that can have characters not representable in Unicode. You can guess that I've learned it the hard way too. And there isn't a way to transfer raw data (of course, you can Base64 encode it etc. but that's not a function of the format, the parties need to agree on format extension in order to understand that).

rwmj• 3 years ago

JSON sucks as a format. You forgot to mention the lack of optional commas, and the need to quote keys.

However 'jq' is absolutely great, and that unfortunately is why we used it for some command line tools that had to output structured data.

bazoom427• 3 years ago

XML does not support non-unicode characters either. For an interchange format this is a good thing, not a drawback.

fluoridation• 3 years ago

And there isn't a way to transfer raw data (of course, you can Base64 encode it etc. but that's not a function of the format, the parties need to agree on format extension in order to understand that).

That seems like an odd complaint. The particular structure of the JSON is also not part of JSON itself, and thus parties need to agree on that too. Why are some implicit agreements okay while others aren't?

planede9• 3 years ago

For some languages null and false are the same thing (and I prefer it that way too)

Ugh, It's hard to pin the blame on JSON for this one. Seriously?

My other gripe is languages/libraries that can't distinguish between missing key and present key with null value. It's more subtle, but these are also different.

acdha5• 3 years ago

Isn't your complain about integers the reverse of your complaint about nulls? I think it's reasonable to want better semantics for precisely specifying common types (I'd add dates to that list) but it seems like in both cases what you want is a way to say exactly what your source and destination types should be rather than having silent lossy conversion.

tragomaskhalos2• 3 years ago

As someone who has circled back into Java land recently after a number of years away, it is startling the extent to which the use of XML in this environment has boiled away to nearly nothing (the only XML file in our projects is the Maven POM), when these languages were once joined at the hip. The addition of annotations into Java has been a game-changer here, but also related ecosystems like OpenApi are using Json and even Yaml in preference.

funcDropShadow2• 3 years ago

but also related ecosystems like OpenApi are using Json and even Yaml in preference.

And that explodes my mind. Yaml is such a steaming pile of shit compared to Json, let alone XML. Yaml is the perfect example for the dichotomy of easiness versus simplicity. At first glance, it looks so easy. No tags, no braces, no commas, you can use comments. It seams to be so much better than XML and Json. After a while you begin to realize that the easiness is only perceived at first glance. In truth it is agglomeration of corner cases that got hyped like crazy. See https://noyaml.com or the top answer here https://stackoverflow.com/questions/3790454/how-do-i-break-a... for examples why I have such a strong opinion of Yaml.

ElectricalUnion3• 3 years ago

My guess is yaml triggers exactly the same initial response as jquery; it's surface seems "small", "simple" and "intuitive", but once you need to actually do things and dig deeper on how it is implemented, you often wish you haven't used it.

planede9• 3 years ago

YAML fucked up bad with 1.1 and YAML 1.2 has virtually no adoption. It could have been great.

62951413• 3 years ago

My impression from perusing a book on Android development is that it feels like XML-oriented programming. It gave me EJB deployment descriptor vibes. A very curious contrast to Kotlin being a modern language.

cryptos• 3 years ago

XML is nowhere near "hip" these days, but I think it is underrated. Yes, a lot of bad things have been done with XML, but the technology itself is just fine and sometimes hated for the wrong reasons.

XML is considered overly verbose, which is not always the fault of XML, but of how it is used.

    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <version>5.9.2</version>
    </dependency>

Could (in principle) be written as:

    <dependency groupId="org.junit.jupiter" artifactId="junit-jupiter-engine" version="5.9.2" />

What really stands out is XML Schema. It is so much more powerful and precise compared to Json Schema.

Hendrikto• 3 years ago

Isn‘t that element vs attribute dichotomy one of the main drawbacks of XML?

You don‘t have that with other formats.

funcDropShadow2• 3 years ago

Somehow this is only mentioned as a drawback of XML, not of HTML --- usually.

HelloNurse3• 3 years ago

It's usually forgotten that XML is intended as a markup language: text is the main content, text is contained in elements, and elements might have properties (i.e. attributes). Elements with no text children are a special case of mixed content, not the norm.

ivan_gammel3• 3 years ago

It’s ambiguous only until you define a design convention for your schema. The rule can be as simple as using attributes only for simple non-user generated values (eg order, postal code, birth date or name is an element, uuid or timestamp is an attribute).

And clients just map whatever defined in schema.

type0• 3 years ago

A lot of bad rep for XML stems from the decades when it was "when all you got is a hammer, everything is a nail"

XML shall be used for Good, not Evil.

pixl97• 3 years ago

The saying I remember from back then is

"XML is like violence: if it doesn’t solve your problem, you aren’t using enough of it."

zaphar5• 3 years ago

XML get's used in a lot of places it shouldn't be used. It also has two competing standards for parsing one of which is far too heavyweight for many of the use cases it has been used in.

9 times out of 10 for data transport you want SAX parsing not DOM. Unless your XML is actually a marked up document DOM is overkill. But many data transport libs used to use DOM which has a very real performance cost. I'm not sure how much of that is still true but the reputation for bloat and being heavy weight was often caused by parsing to DOM and then deserializing from there.

tablespoon• 3 years ago

XML get's used in a lot of places it shouldn't be used. It also has two competing standards for parsing one of which is far too heavyweight for many of the use cases it has been used in.

That's not the standard's fault. DOM and SAX (or rather streaming parsing in general) both have their places, it's up to the developer to know when to use which.

Of course, developers often make bad choices, but the only way a standard can "solve" that is by forcing a developer to make bad choices in some contexts, so it's not the developer's fault anymore.

planede9• 3 years ago

I have some experience with libxml2. I would say DOM parsing is partly inefficient due to the many small allocations, and the library does not offer fine grained control over it. The allocator is globally replaceable, but that's not very flexible.

I wished many times if I could just use an arena with a dumb allocator for a small document and just drop it when I finish with the document, but it's not very easy to do with libxml2, unless you write your own DOM parser on top of SAX.

ben7799• 3 years ago

Lots and lots of applications make the exact same mistakes with Json.

Read the entire document into memory and let Jackson parse it out into a heavyweight object graph. Blech.

hermitcrab• 3 years ago

"Welcome to Niche Software, a MicroISV based in Wellington, New Zealand. "

Ahh, "MicroISV". That is a term I don't hear enough these days!(Greetings from a UK based micro MicroISV)

revskill• 3 years ago

JSX is standard today ?

D13Fd• 3 years ago

The fact that you can use it in two ways, one of which is bad, is not really a good thing.

gjvc2• 3 years ago

Yes! The Caucho Resin server https://caucho.com/ used / uses RelaxNG as its schema, and , amazingly flexibly, allowed for either attribute- or element-based configuration, the latter being much more pleasing to the eye in many cases. I don't think I've seen such flexibility anywhere else.

gjvc2• 3 years ago

oops! I mean "the former" -- attribute-based configuration being much more pleasing to the eye

indymike• 3 years ago

What really stands out is XML Schema. It is so much more powerful and precise compared to Json Schema.

As someone who was involved in making XML based international standards (HR-XML), I think what made JSON great was that it was so simple that there was initially no industry to be made out of making schema. XML had so many ways to do things a schema was needed. JSON? Not so much.

BoppreH• 3 years ago

The problem is that XML is a markup language, which is entirely incompatible with the way data is usually structured in programs.

Your example is simple enough, but consider the following:

    <root>
        This is some text.
        <type1 attributes="yes">
            This is content with <i>markup</i>.
        </type1>
        <type1>
            Another element of the same type.
        </type1>
        <type2>
            And this is an element of an entirely different type.
        </type2>
        This is more text.
    </root>

How the hell can I map that to my data model? My application data will, at worst, be a graph of class instances with named fields, containers (lists, sets, etc), and primitives (strings, ints, etc).

The fact that XML has both attributes and subelements is already a huge impedance mismatch because classes have a single namespace. But what do I do with the repeated element? With the text interspaced with tags? What's the API to navigate this mess?

I'm pretty sure that's why JSON won: it started from the application data model and just simplified it (no custom types, trees only). A deserialized JSON object doesn't even need an API, it's just a nested structure of built-in types found in any modern programming language.

ivan_gammel3• 3 years ago

This is a made-up problem that never existed in reality. On practice if you have to deal with a mix of text and elements, it is always deliberate and explicit choice which probably makes sense. In my decades long career I have never seen it, so it must be a rare case.

You ask about API: XML mapping problem has been solved more than 20 years ago and standard cross-platform API exists since then. Have you ever heard of DOM? Have you ever heard of data mappers? XML Schema-based code generation? In modern Java you can even use the same mapping both for JSON and XML.

There are two reasons why JSON dominates in certain applications: easier to write a small structure, browser-friendly. XML still has advantages in large datasets and schema validation.

thadt• 3 years ago

It's been so long since being part of a debate cycle about whether a piece of data was an 'attribute' or an 'element' in XML. I had forgotten how much angst was eliminated when JSON came along like some kind of uncivilized philistine and just didn't have attributes.

ivan_gammel3• 3 years ago

In JSON everything that is not map or array is effectively an attribute. It is perfectly possible to use the same logic when defining XML schema.

raydev• 3 years ago

It is so much more powerful and precise compared to Json Schema.

Maybe so, but the continuing popularity of JSON, YAML, TOML, and recent competitors shows that developers (in general) prefer something that is both easily readable and editable by hand.

Personal opinion of course, but I wouldn't call XML precise, I would call it the opposite because it's too flexible. The older I get, the more I think programming languages/markup languages/config languages should be opinionated and strict.

fancyfredbot• 3 years ago

It's hard to imagine nowadays how much hype there was around XML. It was the buzzword of the day for a while. It felt as pervasive as "AI" or "blockchain" have been recently. There were many conferences on the subject, every acronym has an X for XML in there (c.f AJAX). All this for a markup language (ok, and a schema/query/transform language). It seems so weird now.

donatj• 3 years ago

Yes! XML was a major feature / selling point of so many industry tools and services. What does it do? No idea, but it uses XML to do it so take my money. So many businesses just screamed about XML. It was honestly kind of a bizarre furor in the early aughts.

oblio3• 3 years ago

Yes! Cryptocurrencies were a major feature / selling point of so many industry tools and services. What does it do? No idea, but it uses cryptocurrencies to do it so take my money. So many businesses just screamed about cryptocurrencies. It was honestly kind of a bizarre furor in the early 2020s.

sk5t2• 3 years ago

Those times were all about XML and application servers like ATG Dynamo, Coldfusion, BEA, JBoss, and WebSphere, with very delicate installation procedures that probably also involved XML.

epberry• 3 years ago

That is crazy to think about. I wonder what this sentence will be about in another 25 years.

otabdeveloper42• 3 years ago

"AI", of course.

api• 3 years ago

The underlying idea was that this self describing format would enable easy interoperability. It wasn’t 100% wrong since before standards like this you had a million ugly bespoke formats and fragile CSV formats.

In the end it was JSON plus API description and generation tooling that delivered, though XML lives on under the hood in lots of places. It’s usually used for cases closer to markup like word processor documents or for very complex configuration files.

Hype is just something you have to get used to in this industry. It’s really bad. Any new thing usually gives birth to a wave of breathless unhinged hype whose main purpose is to get corporate purchasers and VCs to cough up money.

oblio3• 3 years ago

I'd argue that (JSON plus API description and generation tooling) didn't really deliver everything.

XML has schemas and validation as part of the main standard. JSON still doesn't have that. There are some things tacked on that 90% of people don't use.

XML has comments, JSON doesn't (!).

JSON is okish and it's easy to pick up. As Visual Basic, Javascript, PHP, Ruby, Python, Golang have proved, convenience at the start beats almost everything else.

butlerm7• 3 years ago

I am not under the impression that JSON has replaced XML as a data interchange format for general usage at all. It is useful for APIs, problematic for configuration files, and relatively uncommon as a file format. I don't think there is any risk of SVG being supplanted by a JSON equivalent for example, nor most business data interchange formats either.

arp242• 3 years ago

I remember people saying stuff like "if you're not using XML everywhere then you haven't understood XML".

wwarner• 3 years ago

As you work through your XML implementation, you'll have many questions, and the answer to every one of them is more XML.

HdS84• 3 years ago

I am the only one who utterly despises namespaces in xml? Query a document with a namespace? Oh no results? Ah forgot to add the namespace...

I had never a use case for namespaces, and the ones I can envision would have been better served by a property or attribute.

acdha5• 3 years ago

Namespaces are great as a concept: I can take your standard document and extend it without any chance of conflicting in the future if you later a property or element of the same name. This is really important for interchange systems since you inevitably end up with cases where a standard format has almost but not all of the vocabulary you need.

Unfortunately, namespaces as implemented in most XML tools are terrible and that's 100% the fault of the implementers. The problem is basically this:

    <foo xmlns:bar="http://example.org/foobarspec">
       <bar:a href="baaz">hello world</bar>
    <foo>

To a human reading that, you should be able to write a selector like /foo/bar:a or even /foo/a but as an XML parser sees the world bar is actually:

    {http://example.org/foobarspec}a

That's an implementation detail but a shocking percentage of XML library developers did not implement a lookup mechanism to resolve it when given "bar:a".

I would maintain that paying a single developer to make libxml2 suck less would have done more to help XML adoption than everything done by any XML-related standards group combined after Y2K.

PurpleRamen• 3 years ago

First thing I do in any XML-Parser I write: search/replace the namespaces. I know this is awful and impacts speed, but namespaces are so f*** annoying to handle with most parsers. Getting rid of them is the simplest solution for my sanity.

mLuby2• 3 years ago

Here's how JavaScript's XSD validator works:

A (XSD) schema validator for NodeJS that uses Java to perform the actual validation. Why? Because Java can do schema validation and NodeJS cannot. https://www.npmjs.com/package/xsd-schema-validator

schemescape3• 3 years ago

That’s not “JavaScript’s”, that’s just a library someone published on NPM, right?

mLuby2• 3 years ago

Yep. JS doesn't have a standard library so npm is the best we've got.

And speaking of the best we've got, the XSD validator above is the only one I'm aware of on npm—kind of shocking given how many overall libraries there are.

It speaks to the unpopularity of XSDs among web devs, which is ironic since HTML is XML.

schemescape3• 3 years ago

Maybe someone can help me find an obscure XML feature that I’m fairly certain I didn’t imagine:

Is there a way to refer to another part of the XML tree from elsewhere in the document?

I vaguely recall hearing about this feature, but now I can’t seem to find it, even after skimming the spec.

oblio3• 3 years ago

Maybe https://stackoverflow.com/a/29371226/1320143 ?

schemescape3• 3 years ago

That’s it. Thanks!

Edit: id/idref

jillesvangurp• 3 years ago

XML basically came out of SGML which had its origins in GML, which dates back to the 1960s. With XML they sort of dropped all the hard/difficult bits and used it as a simpler serialization format for tree shaped stuff. That happened after people got annoyed with another vaguely SGML inspired thing called HTML which of course was a bit lacking in specification and uniformity at the time. Which is why this whole thing came out of w3.org.

So XML was specified and then right along came XHTML 1.0 and later 1.1, which was just whatever HTML 4 was supposed to be (Netscape, Opera, Microsoft, etc. had some conflicting points of view on this) but in well formed XML shape.

Of course XHTML became a bit of a problem when it turned out that browsers had to deal with mostly not so well formed HTML coming from servers. People were just slapping whatever header on top of their html to trick browsers into various parsing modes. Eventually, the whole notion was dropped with HTML 5 just standardizing all the not so well formed bits in a way that browsers could make sense of. The WhatWG was responsible for that one. When it became clear that was the only HTML/CSS interpretation that still mattered, it was grandfathered into the w3.org and since then people have stopped talking about xhtml as a thing to pursue.

Around the same time the w3c lost the plot a little bit and went off and built a whole sandcastle of standards on top of XML that mostly did not get a whole lot of traction. Capital S Semantic Web vs. lower case semantic web became a thing. One was a new web reimagined on things like SOAP, XHTML, RDF, and whole lot of other things. The other was web 2.0 featuring things like RSS (almost but not quite well formed xml, depending on where you got it), micro formats, hashtags, and a whole lot of people doing rather hacky things with wordpress, php, and related things. Then AJAX became a thing when MS added XmlRpcRequest. Soon, people figured out that this stuff just got a whole lot easier if you send Json over that instead of XML. Much easier to parse from Javascript for example.

goto114• 3 years ago

thing called HTML which of course was a bit lacking in specification and uniformity at the time.

HTML was actually pretty well specified except for one crucial omission: How a user agent should treat invalid HTML. XML simply declared that a user agent should stop processing and report an error - something you can get away with when you don't need to worry about backwards compatibility. Unfortunately this left XHTML dead in the water.

spapas82• 3 years ago

Recently I had to convert a bunch of well structured XMLs to csvs (flattening). I downloaded the apache xalan-c (xslt engine) and xerces-c(xml engine) windows binaries from 2004 (!) and to my surprise they worked fine on windows 10.

After that it was half an hour work to write a quick XSLT and convert the XML to CSV.

I know that there are other (even easier) ways to extract data from an XML but I have to admit that writing an XSL transformer and see if working gives me joy.

Probably I'm getting too old remembering the old days...

bane• 3 years ago

There's lots of comments here on XML as a file format, but that's not precisely correct (even the opening of the wikipedia article on it has the same conceptual flaw in the opening description). I think understanding what it actually is, and why it exists can really offer a lot of perspective. As somebody who lived through the emergence of XML (and HTML) from SGML, and remembers SGML actually in use in some settings, I think it's important.

XML is a specification for how to create data interchange formats using a semantic, hierarchical, organizational scheme -- it's not a file format.

You create data formats that follow XML, but every different schema is a different data format.

What XML did, and the reason it was important, was that it made it much easier to design and specify data formats that had standard tooling for writing, validating, and reading, than pretty much any other earlier conceptualization. Before then, pretty much every piece of software, and even every different version of the same software, had a custom, usually binary, undocumented, and proprietary data format that it used to serialize data (usually as a file format on disk) or to transmit data across a network.

Today, it's more or less assumed that when you serialize data to disk, or build up some data object to send over a network, it'll probably be in some kind of human readable format, figuring out how to encode your data into bits, or dump out serialized C structs or whatever isn't the common practice anymore. Even using binary serialization format like Cap'n_Proto or BSON is considered kind of unusual outside of some large organizations. Developers back in those days were dealing with a variety of CPU and memory architectures, endianness, word size, and other things that made cross-platform movement of data a total nightmare. And almost nobody documented their formats in any way, reverse engineering formats was a valuable skill in some contexts.

There was some precedent for XML-like approaches. A very common one was CSV, which was often used to move tabular data between systems (and is useful to view as another "specification" for making data formats). But again, it lacks a standard specification, lacks validation, and has other flaws that make it frustrating to use for data interchange.

When XML hit, it was kind of like the enlightenment happened. Developers suddenly had tooling and standards that let them not only pump out data formats in a self describing way, but schemas that also let them know they were writing the data correctly and let readers know how to parse it. Even better, a human can usually just open up a data format created to the XML specification in a text editor and just read-off the data fields and values. Long-term archival became easier, networking systems became easier, RPC and API development became almost trivial, and so on. SOAP was created because XML was created. Most of Java's built in data format stuff and RPC functionality assumes XML. Thousand page books written by a dozen authors showed up in stores trying to explain to developers all of the weird edge bits and use-cases.

Then the ugly parts of XML started to settle in. The verbosity, the overengineered bits, the lack of clarity over fields and attributes, stylistic disagreements over how to organize data using it. Turns out most developers don't want to go through the hassle of writing out the validation schemas so that was usually left unused, and then most of the tooling actually wasn't very good -- for a long time good implementations of readers/writers/validators just didn't even exist for many languages. It also turns out just handling and parsing through XML, even with decent tooling, is kind of an arcane chore.

Since the early 2000s then, there's been an explosion of approaches that all have learned the lessons of XML, the good and the bad, and seek to tackle various perceived shortcomings: size, ease of use, etc.

XML is an exquisite crystal castle, built using the purest most elegant thinking, over decades. It turns out most people just want good enough to get it done. So most developers it seems just serialize to JSON or JSONL, gzip it, and send it over the wire or write it to disk. If you're lucky somebody will write down the fields and expected data values in a team wiki somewhere, and off everybody goes. Gone are validators, carefully crafted hierarchical data values, schemas with carefully considered semantics.

As somebody who lived through the dregs of binary formats and the heights of XML, I think this is okay. But it's good to know that XML exists if extreme formalism is still needed, and binary formats are relegated to highly niche use-cases.

acdha5• 3 years ago

I remember being excited about XML when it first came out, since there were so many clunky languages used for configuration and data storage. That feeling lasted for a couple of years, and the 2000s were basically a long disappointment as the parts I liked were buried under huge piles of enterprise ejecta.

The main lessons I've drawn looking back:

1. Tool quality matters more than it might seem, and you can't count on other people to fix your mess for you. A couple of things which come to mind were the lack of a good formatter and validator (people spent too much time on things like formatting errors, especially if they were starting with a very lenient parser and things seemed to work for a while until they hit a standard one) and also the way things like XSLT/XPath shipped major spec updates but since nobody implemented them in libxml2 most developers couldn't use them, use them in browsers, etc.

2. Usability similarly matters far more than experts think: I'm using namespaces here because in my experience that spoiled more people on XML than any other single feature. The requirement that you pedantically specify the namespace in things like XPath or XSLT meant that people couldn't write those using the values they saw in front of them without getting errors or, worse, silent data loss. This reliably tripped up even experienced users and it could have been easily avoided by making a handful of key client libraries smarter. Everyone deep in the XML community has learned and internalized this but it came up over and over and over as something users hated.

3. Continuing that theme, documentation and especially examples matter a lot for the same kind of reasons. I remember about a decade ago getting evangelized by a semweb person and evaluating a couple of the things they mentioned. The only documentation available for any of the half-dozen referenced specs were the specs themselves (which were not written targeting implementers or mutually compatible), and there were no examples which I could find which matched the current versions of the specs, worked with the few public tools, or, in a couple of cases, were even valid XML.

The common thread in all of this was hubris and especially the assumption of inevitability. A lot of people were aware of these as sources of friction but seemed to assume that someone else would make better tools, train people, write good documentation, maintain examples, etc. — especially since if you were the kind of person sitting on a W3C community, these were less of a problem for you since you had the knowledge & skills to keep these from being significant impediments.

The XML spec is 25 years old today

Related Stories