Json ⊄ js

https://medium.com/joys-of-javascript/42a28471221d

by rpsubhub • 14 years ago

216 102 14 years ago

Rauchg2• 14 years ago

Magnus Holm did a great job describing this after he fixed the problem in Rack::JSONP. Good reference as well for those interested in the topic:

http://timelessrepo.com/json-isnt-a-javascript-subset

0x0• 14 years ago

What a shame that \u hack isn't acceptable in javascript, http://code.google.com/p/v8/issues/detail?id=1939#c4

coolj• 14 years ago

> The abstract operation Quote(value) wraps a String value in double quotes and escapes characters within it.

It doesn't look like http://es5.github.com/#Quote has anything to do with serializing to JSON string--am I missing something?

lucian1900• 14 years ago

Petty much all web technologies appear to be made of twine and duck tape. It's so sad that this is in many cases the best we have.

Already__Taken• 14 years ago

That just seems to be what happens when so many people have to collaborate.

Plus its still just mega young. Look at legal systems, they've been around forever and it can be your life's work just to understand them at a competency to participate.

The miracle of software isn't that it works, it's that it does anything at all.

rwmj• 14 years ago

That's ridiculous. JSON is replacing XML which is a well-established interchange technology that doesn't have all the problems of JSON (like no good way to represent 64 bit numbers, weak typing, barely any types in the first place, etc).

protonfish• 14 years ago

That's quite a conclusion to jump to. Maybe you haven't noticed but the Web is running great, especially when developed by thoughtful, competent programmers.

It's sad how many non-web developers still complain about how the Web is sour grapes.

dllthomas4• 14 years ago

It's not that the people working on it haven't been great, it's that there's so much path dependence, it's frustrating. Which is not at all to say that there isn't path dependence in plenty of other domains.

beatgammit• 14 years ago

One more thing that really bugs me about JSON is that it doesn't support Infinity, -Infinity and NaN. Python's JSON library does, which leads to some interesting breakages.

Sure, for NaN you can use null, but for Infinity, you have to use really large/small numbers, which can also lead to other problems.

udp• 14 years ago

If Python's JSON library accepts Infinity by default, that's a bug and should be reported.

enaeseth• 14 years ago

You can try to report it as a bug, but the docs indicate that this is by design.

JulianWasTaken• 14 years ago

It isn't. It's documented behavior. See http://docs.python.org/2/library/json.html. (simple)JSON deviates from the spec in a few minor ways.

fdr3• 14 years ago

Perhaps more pervasive and despairingly problematic is number representation. JSON only, per specification, supports arbitrary precision numeric representation. However, Javascript is -- also per specification (as I understand it) -- entirely floating point. That makes working with money in Javascript somewhat hazardous. While there are various arbitrary precision libraries out there for Javascript to assuage this, the problem is most JSON-parsing routines will always force one through a floating point conversion anyway, so a loss of precision is more or less inevitable.

While swiftly coercing raw off-the-wire number representations to one's arbitrary precision library of choice can avoid most cases of noticeable accumulated error, it is irksome that the only way everyone seems to get by is by cross their fingers that any loss in precision caused by "JSON.parse" is meaningless to their application.

Or, the problem can be soundly avoided by using strings in the JSON payload, which is lame but effective and probably one's best practical choice. It is clearly an example of corruption spreading from Javascript to an otherwise reasonable feature of the data representation format JSON.

mikeash5• 14 years ago

JavaScript uses IEEE754 doubles for numbers, so as long as you can count on all of your parsers to do that, then you can rely on any integer up to 2^54 being represented exactly. For money, work in cents.

chris_mahan• 14 years ago

You mean, for American money, work in cents.

fdr3• 14 years ago

That's faster, but I think can be a poor trade-off because:

1. The speed gain is negligible for most programs

2. The addition of any pricing that requires fractional cents will require careful work to handle unit conversion to maintain integral representation.

#2 becomes ugly when one has an API many people use, and one must bother them to update their code paths to use the new, higher precision that can handle all money as integers. This is not even counting the case where one tries to cut a corner and someone ends up not doing the conversion they ought to.

Also, programs often are more lucid when operating in terms of the frequent units of choice, such as dollars or fractional dollars. Few domains price everything in cents by preference, because in aggregation often dollars -- sometimes many -- are exchanged. The problem gets worse if one needs weird units like 0.1c to regain integral numbers.

There are ways around these, but falling back on strings seems to me the lowest-maintenance option.

On the other hand, integral representations have few dependencies (a compliant javascript interpreter), which is also a pretty big plus for that.

ahoge• 14 years ago

It's 2^53 (9 quadrillion).

Math.pow(2, 53) === Math.pow(2, 53) + 1 // true

Math.pow(2, 53) === Math.pow(2, 53) - 1 // false

mayoff• 14 years ago

Many domains, from gas stations to stock markets, require sub-penny pricing.

beagle33• 14 years ago

The "work in cents" thing usually works for same currency, but not when you involve foreign exchange:

Up until 4-5 years ago, the EUR/USD or GBP/USD or GBP/EUR were quoted with 4 decimals; Nowadays, there are quite a few places where you have to deal with the 5th decimal to make the books balance. It's going to be a roundoff error either way, but it might be a round-off error that keeps been counters, auditors and therefore eventually IT people awake at night.

atesti• 14 years ago

Lua does not provide integers, too. From their mailing list I have learnt that the IEEE rules guarantee, that e.g. doubles have absolutely now problem representing 32bit integers. You write the same here. It seems to not be intuitive, but IEEE took care of it.

But I wonder why you write that for values in 1/100 one should multiply by 100! Wouldn't an IEEE double have no problems with values with two decimal points, too? Or are there many problems with that, e.g. that $0.01 + $0.01 might already cause rounding errors which do not even out by the clever IEEE rules?

Of course 2^54 woudln't be the highest value where one shouldn't worry anymore, but something smaller.

dgreensp• 14 years ago

JavaScript allows all the same numeric literals in its syntax, it just imposes additional semantics on them.

cbhl• 14 years ago

If I remember correctly, this was a problem with Facebook user IDs. I believe they ended up making the value a string in the API to deal with this (and padding new user IDs by some large power of ten so that developers would hit this code-path early enough to catch it).

cmwelsh• 14 years ago

The only real solution for high precision numbers in JavaScript are string-based "big num" classes, unfortunately. Sending important numbers (banking, gas prices, etc) as the built-in number type is just setting yourself up for pain.

JoshTriplett2• 14 years ago

> However, Javascript is -- also per specification (as I understand it) -- entirely floating point. [...] While there are various arbitrary precision libraries out there for Javascript to assuage this

Interesting! I hadn't seen those before.

That strongly suggests that browsers ought to add some built-in extension types for arbitrary-precision arithmetic, using fast native libraries like GMP. That would then allow such JavaScript libraries to use the native types when available, and their existing pure-JavaScript support when not.

SoftwareMaven• 14 years ago

Browsers shouldn't do it. Ecmascript should do it.

andrewcooke2• 14 years ago

anyone else see the strangest mess on view source for that page? (chrome on ubuntu)

[edit: i guess it's confused by the line break characters. i reported an issue.]

[edit2: whoa, wget seems to show the same thing when i look at the source in emacs...]

[edit3: ok, i am an idiot. it's just quotes in a bunch of meta tags. sorry. move along. nothing to see here.]

prezjordan• 14 years ago

I'm curious why every paragraph is named like that.

xmas_project• 14 years ago

The "uniname" command in the uniutils package (apt-get install uniutils) is great for checking "invisible" or weird Unicode characters. `echo '{"str": "own ed"}' | uniname` will show the LINE SEPARATOR (\u002028) hidden in the string.

lacker• 14 years ago

Another problem is unicode characters that are not in the basic multilingual plane. JSON supports those, but many Javascript environments (like basically all browsers) do not.

JulianMorrison2• 14 years ago

The other reason: if you are using a Turing-complete language to eval your nominally context free grammar as if it were code, you are an idiot who is asking to be hacked, whether through quirks like this or otherwise. (This also goes for PHP and ERB.) http://www.cs.dartmouth.edu/~sergey/langsec/occupy/

Rauchg2• 14 years ago

It's actually trivial to make evaluation safe for JSON input as seen in the very short `parseJSON` implementation in jQuery:

https://github.com/jquery/jquery/blob/master/src/core.js#L52...

Also, for the JSONP technique to work, it _has_ to be valid JavaScript so the escaping is necessary.

chimeracoder• 14 years ago

I don't know enough Javascript to give the analogous exploit myself, but that approach looks dangerously similar to some attempts I have seen at making Python's "eval" safe - attempts which, I might add, all fail to capture corner cases that any sufficiently determined or motivated attacker would eventually discover.

ot• 14 years ago

Previous discussion: http://news.ycombinator.com/item?id=2550145

sil3ntmac• 14 years ago

I often pass inline data from server -> client JS using a meta tag. In rails3 it would look like:

However this has always seemed unclean to me. Does anyone else have a better, alternative method of inlining data? I'd rather not use inline scripts for the exact reason they mention.

scalvert• 14 years ago

We use this technique https://www.quora.com/JavaScript/What-is-the-best-way-to-pas...

jasonkostempski• 14 years ago

Can you give it directly to the JS working on that data by setting a var (var x = <%= @data.to_json %>;) or passing it in as a parameter to an initializing function call (APP.init(<%= @data.to_json %>);)? Edit: 1. I hope it's not user generated data. 2. The output in this case wouldn't need to be JSON syntax, it could be object literal notation, saving space on all those double quotes but I doubt you have a helper method available for that.

ricardobeat3• 14 years ago

Use data-* attributes [1], or print it in a script tag with a text/anything type:

    <script type="text/json" id="mydata">
        { data: "..." }
    </script>

    var data = JSON.parse($('mydata').textContent)

[1] https://developer.mozilla.org/en-US/docs/HTML/Global_attribu...

benmmurphy• 14 years ago

If you put it in a script tag you need to ensure you escape </script>. Though, it is probably safer to escape <, >, and & just to be sure.

dotborg• 14 years ago

you can put anything inside class attribute

you can mix actual classes with JSON, just put JSON at the end:

CSS selectors will work just fine

javajosh• 14 years ago

This is a good catch, but the presentation is awfully passive aggressive. Yes, if JSON wants to strictly be a subset of JS, then those two characters need to be treated specially - either excluded from JSON, or specially escaped in JSON libraries. The simplest solution, clearly, is to exclude them from JSON. (You can still use those characters, but you have to escape them). There's no compelling reason not to do this - and I doubt that this will cause a problem anyway.

gurkendoktor• 14 years ago

If you can paste these invisible characters into a chat box to break a website that uses eval(), then someone will inevitably do it, simply because it is the internet.

powrtoch2• 14 years ago

I feel really dense, but I don't understand why the example line is throwing an error. The article mentions line terminators, but it doesn't seem to contain any, and I also don't understand why "owned" would be escaped the way the author says... it looks as though the interpreter is just rejecting the use of quotes around the key. But I'm sure I'm just missing something, so I'd be much obliged if someone could enlighten me.

tjohns• 14 years ago

The line terminator is there, it just doesn't render as anything visible. (It's inserted directly as a unicode character, rather than it's HTML-escaped equivalent, so you won't even see it if you view source!)

However, if you take a hex dump of the page, it becomes quite apparent:

https://gist.github.com/4393892

Note: The file is UTF-8 encoded, so you'd be looking for E2 80 A8 instead of \u2028.

jfarmer• 14 years ago

There's an invisible-to-the eye unicode character in the string "owned," if you copy and paste the text from the website.

JSON is fine with these characters, but JavaScript is not.

For plain-jane JSON this is usually fine, since you're not just evaluating the JSON as JavaScript, but are running the returned data through a JSON parser. A properly-designed JSON parser will escape any JSON-valid-but-JavaScript-invalid characters.

JSONP, however, works differently and will use use the JavaScript parser. Womp womp.

The blog post also lists two other cases, although the first case -- parsing JSON using eval -- is both insecure and incorrect. I haven't seen people do that in ages and ages.

powrtoch2• 14 years ago

That makes sense. I was confused because I was also getting the error when I typed the line into the interpreter by hand. The issue being that I forgot to assign it to a variable (facepalm). Thanks for the explanation.

mhotchen• 14 years ago

Just an fyi, jQuery's core JSON parser actually uses eval (well, new Function(), which is almost the same, but with scope protection).

https://github.com/jquery/jquery/blob/master/src/core.js#L52...

rpearl• 14 years ago

I have to comment on this line:

> Some libraries implement an unsafe, but fast, JSON parse using “eval” for older browsers

eval is not fast! In fact it is the opposite of fast. Most JIT optimizations go away in the face of eval()! Do not use it even if you know it's safe. Use JSON.parse instead.

shod• 14 years ago

"Older browsers" means those that predate native JSON methods. Libraries often use eval() as a fallback.

Xion• 14 years ago

The other thing is that JSON scoffs at trailing comma (both in object and array literals), while JavaScript engines are perfectly happy to accept them.

troels2• 14 years ago

Not all of them are. IE6 will choke.

martin-adams• 14 years ago

And IE7, and IE8.

Toshio• 14 years ago

YAML is the new JSON dontcha know.

michaelmcmillan• 14 years ago

Judofyr has covered this in depth on his blog: http://timelessrepo.com/json-isnt-a-javascript-subset

Json ⊄ js

Related Stories