Parsing JSON is a Minefield

by beefburger • 10 years ago

559 292 10 years ago

I still love JSON regardless :) Client / server side languages have first class support for serialization and in most cases the data structures are rather easy.

I'd be very skeptical if one would suggest an alternative format for a web based project, however I can imagine such situations.

mi100hael2• 10 years ago

In conclusion, JSON is not a data format you can rely on blindly.

That was definitely not my take-away from the article. More like "JSON is not a data format you can rely on blindly if you are using an esoteric edge-case and/or an alpha-stage parsing library." I haven't ever run into a single JSON issue that wasn't due to my own fat fingers or trying to serialize data that would have been better suited to something like BSON.

Tepix2• 10 years ago

You are not considering a hostile environment like the internet where an attacker will use edge cases to get unforeseen results.

inimino16• 10 years ago

The takeaway is "JSON parsing can fail and possibly crash" but I don't think that should be news.

mi100hael2• 10 years ago

I couldn't really care less if someone POSTs some garbage JSON that results in them getting a 500 response. Better than someone POSTing an XML bomb and affecting other peoples' requests. Please enlighten me if you know of a serialization format with libraries for all common languages that lacks any gotchas or edge cases.

empath752• 10 years ago

Yeah, basically the json parsers and generators and the programs that use them seem to have settled on implementations that mostly work together.

jerf• 10 years ago

The sum totality of all the issues raised in that post is not an esoteric edge case, even if each individual element is an esoteric edge case. If you haven't encountered any of them in your real code yet, there's two basic possibilities. Either you aren't using JSON very hard at all... or you have encountered them and you just didn't realize it. You will, sooner or later.

I'm not saying JSON is bad. Personally I think the sloppiness that this post enumerates is part of its success, and part of why it has so much support in so many languages. The more you nail down the semantics, the harder such widespread support gets. Right now you get a nice subset of the JSON spec that works in a huge variety of languages, where the JSON support for a given language usually converts the JSON into something fairly native for the language. This wouldn't be possible if you nailed down the semantics as hard as something like Protobuf does. For instance, a ton of statically-typed languages will let you encode and decode integers into JSON. But JSON doesn't have an integer type. Strict JSON support ought to forbid integers. But it's sloppiness lets us all just sort of ignore that and get on with life.

If you have a need for precision, avoid JSON. And people probably generally need more precision than they realize and probably ought to reach for JSON a bit less often than they do. But on the other hand, the whole thing does mostly work, right? That can't be ignored.

inimino16• 10 years ago

JSON doesn't have an integer type, but it certainly supports integers. Within, obviously, implementation defined limits. I'm with you up to "if you need precision, avoid JSON". Actually JSON is fine for the kinds of precision most use cases require, and when it isn't, you probably know it.

wtetzner2• 10 years ago

In conclusion, JSON is not a data format you can rely on blindly.

Seems like an odd thing to say in general. Perhaps "JSON parsers are not libraries you can rely on blindly", but that seems true of everything.

romaniv• 10 years ago

If there are two servers/services that communicate via JSON and they use different parsers, these types of issues can lead to rather nasty problems. Even if both parties fail gracefully on their own.

This gets even worse if your software is an integration layer between two services you do not control.

indexerror• 10 years ago

In conclusion, JSON is not a data format you can rely on blindly.

What does HN suggest for configuration files (to be written by a human essentially)?

I am looking at YAML and TOML. My experience with JSON based config files was horrible.

FreeFull• 10 years ago

In my personal experience TOML works really well. It's a little reminiscent of .ini files, but definitely is better.

jzwinck• 10 years ago

YAML or TSV depending on whether your configuration looks like a rectangular table.

If you want extreme flexibility using C++ as the main language, take a look at my project: https://github.com/jzwinck/pccl

It lets you configure your C++ apps using Python. Config items can even be Python functions.

kozhevnikov• 10 years ago

https://github.com/typesafehub/config/blob/master/HOCON.md

ronnier• 10 years ago

This is what I use, it's very good.

https://github.com/typesafehub/config

marios• 10 years ago

I don't have a specific recommendation, but when I see a project uses a JSON file as configuration, I wonder: "hasn't the author ever needed to include a comment in the configuration ?".

cestith• 10 years ago

I can't speak for others. When I write something that uses JSON for configuration files those files are pretty much always written from configuration management. The comments are then in the manifest/recipe/playbook that generates the config file, which is where people are actually working with the values.

robert_tweed2• 10 years ago

Funnily enough, as I've been experimenting with Chef and trying to stick to JSON config files where allowed, I was again struck that (a) it's not a good choice for config files (b) it's an OK choice though (c) lots of people are using it anyway (d) nearly everyone that does so (including Chef) allows comments, so in reality are not actually using JSON at all.

Point (d) is the important one. I really think we need a standard for json-with-comments. JSONC or whatever, but it should have a different standard filename and it should have an RFC dictating what is and isn't allowed. Personally I would allow only // comments because there are too many subtle issues with C-style comments, but it may be too late to agree on that.

Half the point of JSON is that if application A stores its data as JSON then application B can parse that without any nasty surprises. Except, there are now probably thousands of noncompliant implementations in the wild that only exist because the standard doesn't allow comments. Each one of those standards adds subtle differences (in addition to the comments themselves) depending largely on how they remove the comments before passing to the standards-compliant JSON parser (assuming they do that, which being DC's recommended approach, is as close to a standard as currently exists).

pilif2• 10 years ago

(b) it's an OK choice though

I really think it's not an OK choice. A config file format that doesn't allow comments provides some of the worst possible UX.

One of the nice things about config files is that normally they are self-documenting, explaining the meaning of the various directives and providing possible values.

Without comments, you have to constantly switch between the documentation and the config file.

Also, the restriction on trailing commas is another really bad issue for a config file language as it pollutes diffs, makes moving lines around needlessly difficult and is one more landmine waiting to happen for the sysadmin editing a file.

No. JSON not at all OK as a config file language.

steveax• 10 years ago

We have had good luck with HOCON for config files:

https://github.com/typesafehub/config/blob/master/HOCON.md

DonHopkins4• 10 years ago

Reflecting the great tradition of "C++", I hereby propose calling it "//JSON".

erik14th• 10 years ago

I like HJSON[0] for files you need to edit manually. You have comments and other user friendly things.

[0]https://hjson.org/

ocschwar2• 10 years ago

JSON with comments? How about { "object":{ "foo":1,"bar":"New Jersey"} , "_comment": "blah blah blah" }

A bit hackish, but always worked for me.

Ketak• 10 years ago

I have actually used Lua before with good success. It was on a smaller scale, so I can't speak to edge cases, but I would certainly recommend considering it at the least.

empath752• 10 years ago

i actually like hcl: https://www.terraform.io/docs/configuration/syntax.html

Ono-Sendai• 10 years ago

XML

isidor32• 10 years ago

Lua was written for exactly this purpose, and I personally enjoy writing with it, so it would be my first choice in most cases.

petre3• 10 years ago

What purpose? Writing config files? Lua is a programming language (a nice one too). It's code. Code should not be used for config files, nor for data serialization because once you eval it you are executing it.

lucb1e7• 10 years ago

What does HN suggest for configuration files (to be written by a human essentially)?

JSON.

YAML confuses many people by being whitespace-sensitive; ini files I find too limited.

DonHopkins4• 10 years ago

A general rule of thumb: Never use yet another non-markup language designed by people who claimed to be designing yet another markup language from the very outset, then after somebody awkwardly pointed out that what they'd designed wasn't actually a markup language, they invent a backronym to contradict that embarrassing historical fact.

It just makes me wonder what the hell they thought they were doing all that time... It's like designing something called YACC, and ending up with an interpreter interpreter!

https://en.wikipedia.org/wiki/YAML

Originally YAML was said to mean Yet Another Markup Language, referencing its purpose as a markup language with the yet another construct, but it was then repurposed as YAML Ain't Markup Language, a recursive acronym, to distinguish its purpose as data-oriented, rather than document markup.

inimino16• 10 years ago

JSON is great for data exchange but config files should be human readable and amply commented. Even rolling your own simple format is probably better than using JSON.

sankyo• 10 years ago

in Clojure we use EDN (Extensible Data Notation) for config files and a data transfer format. https://github.com/edn-format/edn

It is really a pleasure to use compared to JSON and XML. While it may not be as compact as ProtoBuffers, Thrift, or Avro, it is human readable and also valid Clojure code. Libraries are ready available to convert it to JSON.

tantalor• 10 years ago

Protocol Buffers text format: https://developers.google.com/protocol-buffers/docs/overview...

petre3• 10 years ago

YAML is equally horrible and the spec is an order of magnitude more complex. I wasted half an hour trying to spot an error in the ejabberd yaml config, only to find out something trivial was missing. At least JSON has braces even though it's not suitable for configuration files. By all means choose TOML or something else (even ini or java properties files) instead.

crdoconnor• 10 years ago

What was the trivial missing thing?

yes_or_gnome3• 10 years ago

YAML has braces. In fact, it's a super set of JSON. Any YAML parser should be able to parse JSON encoded data with one exception. Block comments. Which is a bastardization of JSON (as mentioned in the article) so block comments shouldn't be a problem in most cases.

steveklabnik2• 10 years ago

I've used JSON and YAML for a very long time, but whenever I have the option, I'll be using TOML every time.

YAML is _huge_ (did you know all JSON is valid YAML?) and those features can come back to bite you... http://blog.codeclimate.com/blog/2013/01/10/rails-remote-cod...

JSON has a number of annoyances, mostly no comments, no trailing commas, all the stuff this article gets into.

Formats like INI or CSV don't really have a spec, or if they do, most implementations don't seem to follow them.

TOML is a bit weird at first, but it's grown on me quite a bit.

slasaus• 10 years ago

After trying json, yaml, json5, java properties, ini and toml, I finally choose hjson* as the configuration file format for the software I'm building. It's the easiest format to read and write IMHO, a bit like nginx config files.

* http://hjson.org

mamcx• 10 years ago

INI files.

Of course, some lunatics try to embed big amounts of text and that is where INI files not look ok.

sirn• 10 years ago

If you decide to use YAML, make sure to check out Strict YAML[1] and its FAQ[2].

[1] https://github.com/crdoconnor/strictyaml

[2] https://github.com/crdoconnor/strictyaml/blob/master/FAQ.rst...

uiri• 10 years ago

I would recommend TOML, but I am a bit biased as the author of the toml python package.

_wiv7• 10 years ago

"NaN and Infinity"

Yeah. And I learned this the hard way with the Perl module JSON::XS. It successfully encodes a Perl NaN, but its decoder will choke on that JSON. (Reported it to the maintainer who insists that is consistent with the documentation and wouldn't fix it)

TazeTSchnitzel2• 10 years ago

Similarly, Python's encoder violates the JSON specification by default, as it produces `Infinity`, `NaN` and `-NaN`, which other JSON parsers choke on.

joesb• 10 years ago

I don't get it. Why would unaware JSON parsers choke on `Infinitiy`, `NaN` or `-Nan`? JSON has no concept of schema.

So if a parser sees "Inifinity", which it doesn't have any concept of, why would it do anything except treating that as string of a word "Infinity"?

rurban2• 10 years ago

Nobody should use JSON::XS anymore, everybody switched to Cancel::JSON::XS which does have those features, and less bugs.

I just added those great testcases, and found previously unknown bugs. But the python test runner from this repo gives a few false negatives, PR coming soon. It was much easier to write it in perl.

benibela2• 10 years ago

Oh no.

My parser/serializer does the same.

How could that be fixed? Anything better to serialize NaN to than NaN? Or parse NaN?

jayd16• 10 years ago

tl;dr JSON with a bunch of shitty extensions is awful. The error handling among JSON parsers is inconsistent.

s_q_b5• 10 years ago

Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention.

Secondly, assume that parsing your input will crash, so catch the error and have your application fail gracefully.

This is the number one security issue I encounter in "security audited" PHP. (The second being the "==" vs. "===" debacle that is PHP comparison.)

As one example, consider what happens when the code opens a session, sets the session username, then parses some input JSON before the password is evaluated. Crashing the script at the json_decode() fails with the session open, so the attacker can log in as anyone.

Third, parsing everything is a minefield, including HTML. We as a community invest a lot of collective effort in improving those parsers, but this article does serve as a useful reminder of a lot of the infrastructure we take for granted.

Takeaways: Don't parse JSON yourself, and don't let calls to the parsing functions fail silently.

lucb1e7• 10 years ago

Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention.

Been there done that. (The medical attention, I mean.) Worked just fine. The article makes it sound extremely difficult, but 100% of the article is about edge cases that rarely happen with normal encoders and can often be ignored (e.g. who cares if you escape your tab character?).

consider what happens when the code opens a session, sets the session username, then parses some input JSON before the password is evaluated

Edit: I responded more elaborately on the unlikelihood of this, but honestly, I can't come up with a single conceivable scenario. How would you decode part of the JSON and only parse the password bit later?

wtetzner2• 10 years ago

Setting the session flag for "this user is logged in" before checking (or even decoding!) the password seems rather backwards to me.

Yeah, that seems like a problem regardless of whether or not you're parsing JSON.

tlrobinson• 10 years ago

How would you decode part of the JSON and only parse the password bit later?

Not saying it's common or likely in this scenario, but there are streaming JSON parsers (like SAX for XML)

nulagrithom• 10 years ago

edge cases that rarely happen with normal encoders and can often be ignored

I've found that "edge cases that rarely happen" are typically 90% of my major headaches.

s_q_b5• 10 years ago

Yeah, the whole idea is a terrible practice, but it is used in industry surprisingly often. People think that PHP's session lock will save them, seeing the potential race condition but not the decode failure.

The scenario is typically username-password-parameters passed as three variables via POST. The offending developer parses all three variables up front for simplicity:

The session user is created from the POST username, the POST password is input sanitized and put in a variable, then the parameters are parsed as JSON. The attacker POSTs a valid user, a valid (but incorrect) password, and malformed JSON.

The login code reads the variables from POST, opens the session, and dies on the JSON decode.

Hint: this bug exists in the wild on a massively deployed framework. I'm working up a responsible disclosure on that one now.

mikeash4• 10 years ago

A parser should never crash on bad input. If it does, that's a serious bug that needs immediate attention, since that's at least a DoS vulnerability and quite likely something that could result in remote code execution. You definitely need to assume that the parser could fail, but that's different. Unless you're using "crash" in some way I'm not familiar with?

herge• 10 years ago

What about raise an exception?

andrewstuart2• 10 years ago

From the context, I'm assuming s_q_b simply means that if given malformed input, the parser should communicate the problem via the appropriate error channels for the given language. In Java, throw an exception; in Go, return an error, etc.

But yeah, the article's example of an XCode segfault is a good example of both poor parsing logic and poor isolation of fault domains. If your json processing library can corrupt the whole process then something's wonky.

steveklabnik2• 10 years ago

This was something I wondered about in these results: what's the definition of "crash"? Specifically, with the Rust libraries, I'm not sure if this means "the program panic'd" or "the program segfaulted", or something else. The former isn't ideal, but isn't the worst. The later would be much more worrysome.

PaulHoule• 10 years ago

Recursion kills.

microcolonel• 10 years ago

Some parsers are dealing with known good input, and should not include validation code for performance reasons. Often you will parse the same JSON many times throughout a pipeline, but you only really need to validate it once. A good example of this is a scatter-gather message bus, where the router parses a message record, scatters it to a large number of peers, and gathers responses. Depending on how latency-critical this system is, it can make sense to validate and normalize the representation at the router, and send normalized (whitespace-free) JSON to the compute nodes.

Manishearth• 10 years ago

In "safe" languages (i.e. ones which aren't supposed to segfault/etc unless there's a compiler/interpreter/VM bug -- PHP, Java, JS, Python all fit in here) I've often heard "crash" being used to just mean an uncaught exception, or something else that brings the program crashing down.

GP says "or catch the error", so they're probably using that term in this sense.

Terr_• 10 years ago

Legacy PHP code has an "Error" failure mechanism where the process just ends or execution jumps straight to a global handler function.

There is ongoing work to deprecate and remove it in favor of actual exceptions, which travel up the stack and are much easier to clean up after.

benibela2• 10 years ago

It can always crash when there is not enough memory.

Especially on the stack.

On of the test cases has 100000 [ and my own JSON parser crashed on them, because it parsed recursively and the stack overflowed. So now I have fixed it to parse iteratively.

It still crashes.

Big surprise. Turns out the array destructor is also recursive, because it deletes every element in the array. So the array destructor crashes at some random point, after the parser has worked correctly.

inimino16• 10 years ago

Parsing HTML is literally orders of magnitude more complex.

It is pretty rare to need to parse JSON yourself (what environment doesn't have that available?) but it isn't that difficult. It's a simple language.

s_q_b5• 10 years ago

Parsing JSON is indeed several orders of magnitude less complex than parsing HTML. In both cases there are excellent parsing libraries, but it would be very unwise to create your own HTML parser for production use, or to use any parser that hasn't seen some serious scrutiny.

jacquesm• 10 years ago

It is pretty rare to need to parse JSON yourself (what environment doesn't have that available?) but it isn't that difficult. It's a simple language.

That, coupled with the fact that it is still so easy to get it wrong and to introduce security issues is exactly what should peak your attention to the seriousness of the subject. Building any parser is fraught with risk, it is super easy to get it subtly and horribly wrong.

ben_jones• 10 years ago

Silent moment for those of us using niche languages to meet production requirements in environments that do not allow third-party code and do not have JSON parsing in the std lib...

dagw2• 10 years ago

what environment doesn't have that available?

Autocad LISP is the one I ran into. At least it didn't 4 years ago. I'm sure there are other niche cases.

Although I'll admit I punted and wrote a trivial 'parser' that was only able to read and write the particular JSON I was dealing with in that project.

favorited• 10 years ago

Last time I worked on a FileMaker DB, it didn't have JSON support. I don't know if that has changed in the last few years.

kafkaesq• 10 years ago

It is pretty rare to need to parse JSON yourself but it isn't that difficult.

In theory, it's not supposed to be "that difficult". But in practice, according to the linked-to article, due to all rot and general clusterfuck-ery in the various competing specifications, apparently it is.

Or do you really think you could wrap your head all around those banana peels, and put together a robust, production-ready parser in a weekend?

edoceo• 10 years ago

The article is about how the parsers behave differently. No suggestions of writing from scratch.

The === issue is in JS as well.

moron4hire2• 10 years ago

> Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention

This attitude really pisses me off. Get off your high horse.

userbinator4• 10 years ago

No kidding. If everyone thought that way, the only JSON parser to exist and which everyone uses would be barely working, because everyone would be put off from trying to write something better.

I have actually written an HTML parser, and it was not so difficult because the HTML spec specifies everything down to the level of tokenising individual characters.

zaphar2• 10 years ago

The implicit subtext here is "for production". It's an excellent rule of thumb since it's highly likely that one already exists that is already battle tested. It's better to use that one. It's not a high horse it's just good horse sense.

He's not talking about rolling your own for fun or learning reasons which anyone should feel free to do.

legulere• 10 years ago

There's nothing wrong with writing a JSON parser, but you shouldn't expect to do anything else for the next time if you want to do it correctly.

Kiro• 10 years ago

Why would you set the session before the password has been evaluated?

jjnoakes• 10 years ago

Why do bugs exist?

RangerScience4• 10 years ago

I had a need to write my own JSON parser for C#, although that was mostly because I hated the data structures the existing C# parsers produced.

I had the advantage that I only needed to use it for rapid prototype projects, and that I could count on all of the data from a single source being the same "shape" (only the scalar values changed, never the keys or objects).

Not following the RFC helped greatly, as I just dgaf about things like trailing commas.

The biggest "gotcha" for my first implementation was a) temporary object allocation and b) recursion depth. The third prototype to use the parser needed to parse, I think it was, a ten thousand line JSON file? The garbage collection on all the temporary objects (mostly strings) meant that version took ~30 seconds. I refactored to be non-recursive and re-use string variable wherever possible, and it dropped down to seconds.

There was a moment in writing it that I thought I would have an actually legitimate use for GOTO (a switch case that was mostly a duplicate of another case), but that turned out not to be the case :/

mort962• 10 years ago

Wouldn't it have been easier to use the C# JSON parser, and then later walk the tree it creates and convert it into a more sane data structure that way?

MaulingMonkey• 10 years ago

Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention.

As someone who has written his own JSON parser, I must concur. Ahh - are there any doctors here...?

In my defense - I was porting a codebase to a new platform, and needed to replace the existing JSON 'parser'. You see, it was:

  - Single-platform
  - Proprietary
  - Little more than a tokenizer with idiosyncrasies and other warts

Why was it chosen in the first place? Well, it was available as part of the system on the original platform. Not that I would've made the same choice myself. We had wrappers around it - but they didn't really abstract it away in any meaningful manner. So all of it's idiosyncrasies had leaked into all the code that used the wrappers. In the interests of breaking as little existing code as possible, I wrote a bunch of unit tests, and rewrote the wrapper in terms of my own hand rolled tokenizer. Later - either after the port, or as a side project during the port to help out a coworker (I forget) - I added some saner, higher level, easier to use, less idiosyncratic interfaces - basically allowing us to deprecate the old interface and clean it up at our leisure. This basically left us with a full blown parser - and it was all my fault.

Takeaways: Don't parse JSON yourself, and don't let calls to the parsing functions fail silently.

I'd add to this: Fuzz your formats. All of them. Even those that don't receive malicious data will receive corrupt data.

Many of the same problems also affect e.g. binary formats. And just because you've parsed valid JSON doesn't mean you're safe. I've spent a decent amount of time using e.g. SDL MiniFuzz - fixing invalid enum values, unchecked array indicies, huge array allocations causing OOMs, bad hashmap keys, the works. The OOM case is particularly nasty - you may successfully parse your entire input (because 1.9GB arrays weren't quite enough to OOM your program during parsing), and then later randomly crash anywhere else because you're not handling OOMs throughout the rest of your program. I cap the maximum my parser will allocate to some multiple of the original input, and cap the original input to "something reasonable" (1MB is massive overkill for most of my JSON API needs, for example, so I use it as a default.)

TeeWEE• 10 years ago

Takeaway: Dont parse json. Every json parser is wrong. Json is not a standard.

Parsing everything is NOT a minefield. We have BNF's parser theory etc for that. Lots of languages have clear unambigious definitions... Json clearly not. ITs a disgrace for the software engineering community.

rhaps0dy• 10 years ago

JSON does have a BNF definition. http://www.json.org/

rurban2• 10 years ago

Oh my, this is pure nonsense.

First, parsing JSON is trivial compared to other parsing tasks. There are no cycles as in YAML or other serializers, it's trivial forward scanning, without any need to tokenize or backtracking.

Second, JSON is one of the simplest formats out there, and due its simplicity also its most secure. It has some quirks and some edge cases are not well-defined. But with those problems you can always check against your local javascript implementation and the spec, just as OP did.

I know very few JSON parsers which actually crash on illegal input. There are some broken ones, but there are much more broken and insecure by default YAML or BSON parsers or language serializers, like pickle, serialize, Storable, ...

Parsing JSON is not a minefield, parsing JSON is trivial.

Takeaway: Favor JSON over any other serialization format, even if there are some ill-defined edgecases, comments are disallowed and the specs are not completely sound. The YAML and XML specs are much worse, their libraries horrible and bloated.

JSON is the only secure by default serializer. It doesn't allow objects nor code, it doesn't allow cyclic data, no external data, it's trivial, it's fast.

Having summarized that, I'm wondering why OP didn't include my JSON parser in his list, Cpanel::JSON::XS, which is the default fast JSON serializer for perl, is the fastest of all those parsers overall, and is the only one which does pass all these tests. Even more than the new one which OP wrote for this overview STJSON. The only remaining Cpanel::JSON::XS problem is to decode BOM of UTF16 and UTF32. Currently it throws an error. But there are not even tests for that. I added some.

Regarding security: https://metacpan.org/pod/Cpanel::JSON::XS#SECURITY-CONSIDERA...

Tepix2• 10 years ago

The important lessen is that you can't blindly rely on your JSON parser to save your ass when you are dealing with untrusted input.

If sending 1000 "["s will crash your application, you have a problem.

I hope the JSON parser authors will improve their parsers.

newsat13• 10 years ago

Anyone else seeing forbidden?

Forbidden

You don't have permission to access to this document on this server.

gt2• 10 years ago

yes, same here

lucb1e7• 10 years ago

You are not the only ones, but it works for me. Google cache link is posted elsewhere in the thread: https://news.ycombinator.com/item?id=12797047

GuzmanMan• 10 years ago

I saw that, but when I refreshed the page I was able to load the content.

beefburger2• 10 years ago

Too heavy CPU load.

Now I turned the PHP page into a static HTML one.

Configuring web servers is a minefield :)

metafunctor2• 10 years ago

The page has been taken down for some reason (getting a 403).

Google cache: http://webcache.googleusercontent.com/search?q=cache:8jVuBmx...

lucb1e7• 10 years ago

Works for me.

Edit: But you are not alone. Elsewhere in the thread: https://news.ycombinator.com/item?id=12797032

metafunctor2• 10 years ago

Yep, looks like it was down only for a few minutes.

beefburger2• 10 years ago

Too heavy CPU load.

Now I turned the PHP page into a static HTML one and it's back online.

Configuring web servers is a minefield :)

andrewvijay• 10 years ago

Bad thing to read when I'm writing a sass to json module

lucb1e7• 10 years ago

The encoding takeaway seems simple: escape everything with \uxxxx characters that is outside of the ASCII range /[ -~]/ (regex) and you'll be pretty much fine. Set the encoder to utf-8, don't leave [dangling,commas,], and a few other things that are obvious from json.org.

inimino16• 10 years ago

Escaping everything outside of ASCII is terribly human-unfriendly and wasteful in bytes as well.

nathancahill• 10 years ago

amelius• 10 years ago

One of the biggest flaws of JSON is that it doesn't support "undefined". This makes translating Javascript structures to and from JSON actually not preserve the original value. Sigh.

indubitably• 10 years ago

But undefined is specific to Javascript… there are lots of other Javascript things that JSON doesn't handle either, like Set or Map objects. It's not intended to serialize arbibtrary JS objects — it's intended to serialize a useful least-common-denominator which has proven useful through experience.

inimino16• 10 years ago

There are lots of reasons beyond this one why arbitrary JS values can't roundtrip through JSON. This is a conscious design decision, not an oversight.

ohstopitu• 10 years ago

When I didn't know better, I wrote my own JSON parser for Java (it was years back and I didn't know about java libraries). From experience: DON'T. DO. IT.

That said, if you have decided to do it....

1) know fully well that it'll fail and build it with that assumption.

2) Please, please, please...give useful error messages when it does fail or you'd be spending way too much time over something simple.

SloopJon2• 10 years ago

Figures that something like this would be posted on my day off. I put this through a parser that I cover, and found that the only failures were for top-level scalars, which we don't support, and for things we accept that we shouldn't. I'll look through the latter tomorrow, as well as the optional "i_" tests.

Test suites are a huge value add for a standard, so thank you, Nicolas, for researching and creating this one. I was surprised that JSON_checker failed some of the tests. I use its test suite too.

mjpa3• 10 years ago

Wrote my own JSON parser (https://github.com/MJPA/SimpleJSON) a while ago... not sure how it's a minefield unless I'm missing something?

valarauca12• 10 years ago

Did you read the post?

The JSON standard(s) are very simple. But in practice this simplicity leads to a lot of edge cases. Very deeply nested structures, numbers that run on to infinity. The point of this post is testing various parsers against these malicious structures.

See this image: http://seriot.ch/json/pruned_results.png

mjpa3• 10 years ago

"In conclusion, JSON is not a data format you can rely on blindly." - that suggests the format is bad when it's highlighting problems with the parsers. I see it like saying "Plain text is a bad format because notepad bails on large files"

SloopJon2• 10 years ago

Perhaps you'd be interested to know that your JSONDemo program fails the following tests:

hang:

y_number_huge_exp.json

segfault:

n_structure_100000_opening_arrays.json

n_structure_open_array_object.json

fail:

n_number_then_00.json

n_string_unescaped_tab.json

n_structure_capitalized_True.json

mjpa3• 10 years ago

I would :)

Was going to run the tests myself later but needed a box with python3 on it!

Confusion2• 10 years ago

There was a great article at some point that explained why 'be liberal in what you accept' is a very bad engineering practice in certain circumstances, such as setting a standard, because it causes users to be confused and annoyed when a value accepted by system A is subsequently not accepted by supposedly compatible system B. Leading to pointless discussions about what the spec 'intended' and subtle incompatibility. Anyone know what article I mean?

webmaven• 10 years ago

Perhaps "The Harmful Consequences of Postel's Maxim"?:

https://news.ycombinator.com/item?id=9824638

If not, perhaps one of these:

http://programmingisterrible.com/post/42215715657/postels-pr...

https://bitworking.org/news/There_are_no_exceptions_to_Poste...

http://trevorjim.com/postels-law-is-not-for-you/

Confusion2• 10 years ago

Yes, thank you, it was the one by 'programmingisterrible' and the linked paper by Patterson, Sassaman, and Bratus.

inimino16• 10 years ago

I've seen this argument most frequently made with regards to XML.

collyw• 10 years ago

Thats pretty much my experience when building software as well. A lot of the time I have been liberal to incorporate legacy data, and every time in has ended up being the cause of the majority of bugs in the systems I have built.

peatmoss• 10 years ago

What ever happened with EDN (pronounced "eden") from the Clojure people? https://clojure.github.io/clojure/clojure.edn-api.html https://github.com/edn-format/edn

I always thought that seemed like a nice alternative data format to JSON. Anyone using this it in the wild?

arohner• 10 years ago

Clojure programmers use it everywhere. I suspect almost nobody else does though.

tragic• 10 years ago

Even if you're a clojure shop, you've got the issue that at your system boundary, everything else the world accepts json and/or XML, and nothing supports EDN/transit. So your data needs to be serialisable to one of those anyway, at least if it crosses the boundary. There is such a thing as a network effect, even in data serialisation formats...

grayrest• 10 years ago

It got supplanted by Transit [0] as an interchange format. Both are only really used within the Clojure community. I use Transit for internal-facing APIs.

[0] https://github.com/cognitect/transit-format

dep_b2• 10 years ago

Now the mess that is called JavaScript dates has crept into any system imaginable in the world. I can understand we needed to go for the lowest denominator but Crockford's card really could cram in another line with a date time string format.

inimino16• 10 years ago

I think adding a date type would have probably doubled the complexity of JSON and taken it right out of the sweet spot that has made it popular.

niftich• 10 years ago

This. Datetime is hard, because RFC 3339 is underspecified [1], and TOML wrestled with this a lot.

[1] https://news.ycombinator.com/item?id=12364393#12364805

petre3• 10 years ago

Just use epoch or ISO1601.

dep_b2• 10 years ago

You're preaching to the choir here. Hell is other people's code.

halomru• 10 years ago

I thought strings with ISO8601-encoded dates was the de-facto norm (i.e. "20161026T2044+0200 for this comment's timestamp).

Have I been living in a bubble with the majority being less sane?

metaloha• 10 years ago

Am I wrong in seeing that PHP seems to fail the least weirdly in the full results?

TazeTSchnitzel2• 10 years ago

It's more complicated in practice. Prior to PHP 7.0, the official JSON extension was replaced with a completely different one in many distributions due to licensing issues, and since PHP 7.0, PHP's official JSON extension is yet another completely different implementation.

DanielRibeiro• 10 years ago

Wow! This was a great practical analysis of existing implementations, besides a great technical overview of the spec(s). Thanks for open sourcing the analysis code[1], and for the extended results[2]

[1] https://github.com/nst/JSONTestSuite

[2] http://seriot.ch/json/parsing.html

edem3• 10 years ago

    JSON is the de facto standard when it comes to (un)serialising and exchanging data in web and mobile programming.

I disagree. Take protobuf for example. You get schemas, data structures, and a parser in one package which is actually a lot smaller than JSON and compiles to nearly all the commonly used languages. Ever since I've started using it my life became so easier! If you don't want your data to be human readable (which is very common) you should not use JSON as a data interchange format.

djur• 10 years ago

JSON is still the de facto standard, regardless of whether it should be.

iamatworknow• 10 years ago

Please tell that to the multi-billion dollar company I'm currently working to integrate with that decided 2016 was a good year to implement a brand new SOAP API.

edem3• 10 years ago

I don't think so. Can you back it up with facts?

cbhl• 10 years ago

In practice, people use increasingly smaller subsets of JavaScript to transmit data.

For example, a common pattern is to transmit (numeric) user IDs as strings so that they don't get mangled by floating-point precision issues with large numbers. You see both Twitter and Facebook APIs do this, for example.

Senji• 10 years ago

Modern languages should feature a GUID type in the standard library.

zodiac• 10 years ago

I've read that you should treat IDs as strings anyway because you want to discourage incrementing IDs, adding IDs together etc. Anything else you want to do with integer IDs, e.g. comparing, can be done with sting IDs as well.

novaleaf• 10 years ago

when parsing human constructed JSON, use JSON5 for the win.

http://json5.org/

kstenerud• 10 years ago

I did write my own parser, but for a reason: I need it to be able to recover as much data as possible from a damaged, malformed, or incomplete file.

Turns out that a good chunk of these tests are for somewhat malformed, but not impossible to reason about files. Extra commas, unescaped characters, leading zeroes... I'd rather just accept those kinds of things rather than throw an error in the user's face. It's a big bad world out there, and data is by definition corrupt.

And this is borne out when I plug my parser into this test suite: Many, many yellow results, which is exactly how I want it.

redleggedfrog• 10 years ago

Crockford needs to write "JSON, The Good Parts."

seagreen5• 10 years ago

No he doesn't. "JSON, the good parts" is just JSON. The problems the post is describing have to do with RFC 7159 allowed extensions.

realkitkat• 10 years ago

If JSON is comparable to minefield, then I guess XML and ASN.1 are nothing short of nuclear Armageddon in complexity and ones ability to shoot themselves into the leg ;-)

Impossible• 10 years ago

Also, as someone who has written an XML parser, according to some of the comments in this threads I'm way beyond medical help, and should give up on life :).

lameexcuse333• 10 years ago

Depends on when you did this crazy thing. Was it in the dark days before they were "ubiquitous"? (for lack of a better word. also it's just a fun word)

Then you're a hero to those who made use of it.

If someone were to do that today, though, then yes, seek help.

;)

ocschwar2• 10 years ago

Well, every person you spared from that experience owes you a beer...

twic• 10 years ago

I got dinged in a thread the other day for writing my own JSON parser, so i'm not sure i should confess to also having written enough of an ASN.1 parser to deal with certain PEM key formats:

https://github.com/pivotal/cf-env/blob/master/src/main/java/...

xamuel• 10 years ago

Have to take HN anti-parser comments with a grain of salt.

I wrote a parser for a format that everyone here would crucify me for. When I went on the job market, I removed it from my github, fearing employers would see it as a black mark. Soon after I got email from the creator of a YUUUGE project (as in, 5 digits of stars on github) asking why I had removed this parser that they were using ...

vhost-• 10 years ago

You definitely can't rely on it. Just the other day I was given a task to take a request payload from our front end and do some stuff with it on our backend. The payload looked like this: {"thing": [{"values": ["foobar"], "type": "blah blah"}, "some identifier"], "other thing": "some string"}. It's mixing types in arrays which is problematic for most statically types languages.

Tips for Go: Don't use map[string]interface{} and circumvent the type system (I've seen this a lot in production). The fix involves the UnmarshalJSON and MarshalJSON interfaces. This lets you put the data into a structure that's sane and re-encode it back to something the other system expects.

austincheney2• 10 years ago

Writing parsers is hard and takes some experience, but its not as hard or as impossible as most of these comments make out. JSON is retarted simple to parse, even in the face of certain edge case ambiguities.

I can say this from experience after having written an HTML/XML parser that provides support for various template schemes: Twig, Elm, Handlebars, ERB, Apache Velocity, JSP, Freemarker, and many more. I have written a JavaScript parser that supports React JSX, JSON, TypeScript, C#, Java, and many more things.

In years I have been programming I frequently hear whining like, "its too hard". Don't care. While you are wasting oxygen crying about how hard life is somebody else will roll a solution you will ultimately consume.

aikah• 10 years ago

where are all these goodies you boast about then ?

austincheney2• 10 years ago

http://prettydiff.com/

paulddraper3• 10 years ago

Lots of issues are trivially answered.

---

Scalars..In practice, many popular parsers do still implement RFC 4627 and won't parse lonely values.

Right. RFC 7159 expanded the definition of a JSON text.

A JSON text is a serialized value. Note that certain previous specifications of JSON constrained a JSON text to be an object or an array.

If RFC 7159 wasn't different from 4627, there'd be no reason for 7159. Same with RFC 1945 and 7230 for HTTP. (Of course, HTTP is versioned...maybe he just means to repeat the earlier versioning criticism.)

---

it is unclear to me whether parsers are allowed to raise errors when they meet extreme values such 1e9999 or 0.0000000000000000000000000000001

And then quotes the relevant part of the RFC 7159 grammar with answers the question:

This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754-2008 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available.

Parsers may limit this however they like. And so may serializers. This includes yielding errors. (Though approximating the nearest possible 64-bit double is IMO the better choice.)

---

So yeah, in the end there is fair amount of flexibility in standard JSON.

To summarize:

An implementation may set limits on the size of texts that it accepts.
An implementation may set limits on the maximum depth of nesting. [this one was never mentioned though]
An implementation may set limits on the range and precision of numbers.
An implementation may set limits on the length and character contents of strings.

Most implementations on 32-bit platforms will not parse 5GB JSON texts.

tofupup• 10 years ago

it is an improvement but when i am in these situations i usually grab the first library out there.

eridius• 10 years ago

Speaking as someone who wrote a JSON parser, this article and the accompanying test suite looks to be very valuable, and I will be adding this test suite to my parser's tests shortly.

That said, since my parser is a pure-Swift parser, I'm kind of bummed that the author didn't include it already, but instead chose to include an apparently buggy parser by Big Nerd Ranch instead. My parser is https://github.com/postmates/PMJSON

ninjakeyboard• 10 years ago

unless for fun, rolling your own json parser is like writing bubble sort for use in your prod app.

youdontknowtho• 10 years ago

You know what is more like a minefield...a minefield...

http://www.afghan-network.net/Landmines/

Not trying to be a dork, but thought this would be a good place to bring up...if anyone is interested...in the usage of landmines in current conflicts and the way that they tend to linger.

Call this a comment factoid. Off topic, but interesting.

michaelp983• 10 years ago

There are alternatives to JSON that are OS, available in many languages, actively supported, and both CPU and memory efficient.

FlatBuffers & Netflix https://www.youtube.com/watch?v=K_AfmRc-TLE&feature=youtu.be...

singularity2001• 10 years ago

JSON = require('json5')

And you can even use comments!! (no comment)

// http://json5.org/

RangerScience4• 10 years ago

This is fantastic. However, it looks like the detailed conclusion is "exactly matching the RFC is a minefield".

About a month ago (for the third time, since I don't own the first two implementations) I made a very forgiving (and very error-unprotected) JSON parser: https://github.com/narfanator/maptionary

The core of JSON parsing, from that experience, seems really simple; it's catching all the edge cases that's hard.

In any event, I look forward to taking the time to test against this test suite!

RangerScience4• 10 years ago

Do you have an explanation anywhere of why each (or any) of the edge cases is supposed to succeed or fail, or why it commonly does what it's not supposed to do?

I realize that's almost as much work as writing each test case in the first place, but even a subset of the test cases having that explanation would be valuable.

nitwit005• 10 years ago

People tend to screw up the unicode aspects more than the general parsing. And, indeed, the example JSON parser provided checks for a UTF-8 byte order mark, but doesn't validate that the data is valid UTF-8, so it will let through strings that might cause an application problems.

Although there is a commented out method to validate a code point, so I guess he understood that it was an issue.

gcirino424• 10 years ago

The correct answer to parsing JSON is... don't. We experimented last hackday with building Netflix on TVs without using JSON serialization (Netflix is very heavy on JSON payloads) by packing the bytes by hand to get a sense of how much the "easy to read" abstraction was costing us, and the results were staggering. On low end hardware, performance was visibly better, and data access was lightening fast.

Michael Paulson, a member of the team, just gave a talk about how to use flatbuffers to accomplish the same sort of thing ("JSOFF: A World Without JSON"), linked in this thread: https://news.ycombinator.com/item?id=12799904

dvt• 10 years ago

Not sure what your point is (or the point of that presentation, for that matter).

Of course there are binary serialization formats that are faster than XML or JSON, and of course they're less error-prone. This has been known for about 40 years now.

JSON/XML are used precisely because people want a human-readable interchange format. For high-performance uses, consider Google's Protocol Buffers or Boost::serialize. You're acting like you just hackathoned the biggest thing since sliced bread, but that's exactly how payloads have been sent (until high-bandwidth made us all lazy) since the inception of the Internet.

gcirino424• 10 years ago

I thought my point was clear - don't get involved parsing JSON; I agree with the OP, parsing JSON is a minefield. I went further by implying that it is also unnecessary when ease of reading isn't needed, and called out some alternatives. I think it's amusing that you mentioned protocol buffers - were you aware that when I mentioned flat buffers that they were built in relation to performance inefficiencies in the very protocol buffers that you mentioned?

We didn't just "hackathoned the biggest thing since sliced bread", btw, we took a real world example of exchanging a human readable format for a human-with-tools-readable one and saw a significant win. High-bandwidth also isn't as prevalent as you think, and yes, you're generally paying both performance wise and occasionally monetarily for the laziness you mentioned. But then, if you've known this for 40 years and don't know how to measure it, there's not much I can do for you in a comment.

memoryhole• 10 years ago

I believe that is the point. Choose the right serialization strategy to fit the job. Most projects default to JSON regardless of how suitable. At some scale that should be revisited since the human-readable / performance trade-off equation can change.

userbinator4• 10 years ago

From experience, I think the whole "human-readable" idea is a bit overrated. All it means is that the format is entirely/mostly in ASCII. But if you have a hex editor, like all good programmers should, binary formats are not any less human-readable (or writable) nor more difficult to work with; and for some, even a text editor with CP437 or some other distinctive SBCS will suffice after a while. It's somewhat like learning a language; and if you are the one developing the format, it's a language that you create.

Then again, I grew up working with computers at a time when writing entire apps in Asm/machine language was pretty normal as well as other things which would be considered horribly impossible by many developers of the newer generation, and can mentally assemble/disassemble x86 to/from ASCII, so my perspective may be skewed... just a tiny little bit. ;-)

ricardobeat• 10 years ago

You may want to take a look at Protocol Buffers or the new Ion format by Amazon; the latter can seamlessly switch between binary and human-readable.

gcirino424• 10 years ago

Yup, I'm a fan of protobufs and the more recent flatbufs. Definitely like the tooling around being able to seamlessly switch for readability. I'll check out ion.

userbinator4• 10 years ago

Unfortunately one often has to interact with APIs which require JSON, but I completely agree that pure binary formats are far simpler to work with.

z3t42• 10 years ago

I want to use something like flat buffers in NodeJS for optimizing websocket traffic and implementing a FS database. But I cant find much stuff for it in JavaScript. Do you (de)serialize the flat buffers or use them directly by abstracting get/set for example via Object.defineProperty ?

gcirino424• 10 years ago

Google included a complete example of how to (de)serialize data with them, as well as generating the accessor functions for JS. See https://google.github.io/flatbuffers/flatbuffers_guide_use_j...

No, I don't try to decode them directly if I can avoid it. We handrolled a C-based byte packer for our honeybadger project only because function access is relatively slow on the interpreter we have on low end TVs, but reading blocks through a Uint8Array is pretty fast. Writeup is here: http://alifetodo.blogspot.com/2016/05/project-honeybadger-pi... . I can push the C packer to github if there's interest, but since you mention NodeJS, you might have better luck with Paulson's benchmark NodeJS example: https://github.com/michaelbpaulson/flatbuffers-benchmarks

SFJulie• 10 years ago

By sheer randomness I was having the thought about it today: I made some code to highlight where the stdlib json module sees the mistakes in JSON decoding in the python stdlib.

I used the exception with string "blabal at line x, col y, char(c - d)" to actually highlight (ANSI colors) WHERE the mistake were.

https://gist.github.com/jul/406da833d99e545085dac2f368a3b850...

I played a tad with it, and the highlighted area for missing separators, unfinished string, lack of identifier were making no sense. I thought I was having a bug. Checked and re-checked. But, No.

I made this tool because, whatever the linters are I was always wondering why I was not able to edit or validate json (especially one generated by tools coded by idiots) easily.

I thought I was stupid thinking json were complex.

Thanks.

77pt77• 10 years ago

it won't parse u-escaped invalid codepoints: ["\ud800"]

How is this not expected behaviour?

The string is not well-formed.

Same thing with decent XML parsers. They croak when you give them invalid codepoints.

dgreensp• 10 years ago

An informative article. The point is not that parsing JSON is "hard" in any sense of the word. It's that it's underspecified, which leads to parsers disagreeing.

Although the syntax of JSON is simple and well-specced:

* The semantics are not fully specified

* There are multiple specs (which is a problem even if they are 99% equivalent)

* Some of the specs are needlessly ambiguous in edge cases

* Some parsers are needlessly lenient or support extensions

chrismarlow9• 10 years ago

this is kind of a vulnerability developers wet dream, especially that graph...

RX14• 10 years ago

Fixes for many of the issues raised by this post have since made it into crystal master: https://github.com/crystal-lang/crystal/commit/7eb738f550818...

devmunchies• 10 years ago

I love how quickly things move in crystal right now. ️

nickpsecurity• 10 years ago

Crap like this is why people should just use older ones that work. Some of the issues I see in the comments weren't present in Sun's XDR:

https://tools.ietf.org/html/rfc4506

Or even LISP s-expressions if you want organized text.

boggydepot• 10 years ago

So what's the alternative? If you had a time machine and went back in time, what would you recommend/bring as an alternative to JSON?

rahrahrah• 10 years ago

Oh boy this is another one of those threads..

Party A: X is harder than it looks

Party B: X isn't as hard as you're making it look

Boring.

Parsing JSON is a Minefield

Related Stories