Floating Point in the Browser, Part 1: Impossible Expectations

https://randomascii.wordpress.com/2020/09/27/floating-point-in-the-browser-part-1-impossible-expectations/

by nikbackm • 6 years ago

64 64 6 years ago

randomascii.wordpress.com

Anyone who's ever worked on programming language implementation will recognise that you will for all time get a steady stream of occasional bug reports that your floating point numbers aren't arbitrarily precise.

Some language implementation bug trackers have a warning not to submit bugs for it front and centre because they get it so often.

saagarjha2• 6 years ago

In JavaScript I assume it’s worse because most people would assume the existence of some sort of integer type but the number silently “becomes a floating point” instead, which can be confusing.

Gibbon1• 6 years ago

I don't use JavaScript but JavaScript using binary floating point always makes me shake my head in disbelief. You got a language designed from the start to deal with user input that can't represent decimal numbers and fractions.

SAI_Peregrinus2• 6 years ago

JSON Numbers are arbitrary precision floating point. JavaScript numbers aren't necessarily numeric, they can be NaN or INF. That disconnect isn't a bug, but can cause problems if you naively convert from one to the other without handling the edge cases.

Spivak3• 6 years ago

The problem is we’re now at the point where we have mountains of code that doesn’t handle the edge cases and very little that can be done outside of being super conservative about what send an unknown JSON parser.

umvi4• 6 years ago

TIL JavaScipt doesn't have an integer type and uses double under the hood to store integers. Yet another footgun to add to the massive stack.

chrisseaton4• 6 years ago

Double represents consecutive integers up to 2^53 - that's even larger than Java's int so it's more than fine for almost all purposes.

umvi4• 6 years ago

I can't help but compare JS to python though. For every footgun in JS, the same footgun does not exist in python including this one.

pmarreck2• 6 years ago

While I think everyone can agree this is just yet another terrible design decision in JavaScript, at least there are libraries to give you this, and it looks like BigInt will eventually be native to JS:

https://github.com/peterolson/BigInteger.js/

mschuetz3• 6 years ago

While I think everyone can agree this is just yet another terrible design decision in JavaScript

I disagree. Double was the best choice they could have made for a dynamically typed language.

conradludgate• 6 years ago

Not quite true. They do use an integer data representation if the number is an integer, it just silently changes to a float if its necessary. So `let x = 2;` the type would be 'number' but the data would just be an integer.

Someone• 6 years ago

That’s an implementation detail.

JavaScript, the language, doesn’t (AFAIK; I don’t keep track of this rapidly moving target) have integers (but does have integer arrays, nowadays. See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Type...)

If the type were integer,

  let x = 2
  let y = x / 8

would have y = 0, but its value will be 0.25.

evilpie• 6 years ago

JavaScript also has newish BigInt support: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe.... It even has syntax: 42n.

mschuetz3• 6 years ago

Given that it's a dynamic scripting language, I think double was the best choice they could have made. It gives serious number crunching performance, an integer range up to 2^53, and, of course, double precision floating point math. The only better choice they could have made would have been to make it static/strongly typed with various numeric types.

pmarreck2• 6 years ago

I avoid FP as much as possible precisely because of these nondeterministic (at least in the practical sense) results. I use integers almost exclusively and if I need fixed point I have view code that renders things correctly (such as for currency). When Erlang didn't have a power function that didn't use floating-point (and thus resulted in rounding errors), I wrote my own: https://github.com/pmarreck/elixir-snippets/blob/master/math...

recursive5• 6 years ago

In this case you're just trading for overflow.

saagarjha2• 6 years ago

What’s also interesting is how the “minimum representation needed to roundtrip a number” is actually done. There is recent (2010) research into doing this efficiently using an algorithm called Grisu: https://dl.acm.org/doi/10.1145/1809028.1806623. Many programming language implementations use it now for this purpose.

ygra2• 6 years ago

Actually, there's even more recent research into doing this really efficiently, using an algorithm called Ryū.

[e] https://github.com/ulfjack/ryu

[-4] https://www.youtube.com/watch?v=kw-U6smcLzk

[1.73] https://www.youtube.com/watch?v=4P_kbF0EbZM

pklausler• 6 years ago

The f18 Fortran compiler has a novel algorithm (calculating in base 10^16) for conversions that is fast and doesn't require big look-up tables.

https://github.com/llvm/llvm-project/tree/master/flang/lib/D...

SAI_Peregrinus2• 6 years ago

I've mentioned it before, but JSON Numbers are arbitrary-precision floating-point. They can't be naively stored as JavaScript numbers in all cases, since JS numbers are IEEE754 double-precision floating point. Likewise, you can't store arbitrary JS number values in JSON Numbers, since it doesn't handle IEEE754 exception values (qNaN, sNaN, ±INF).

So if you're trying to publish data from a GPS chip that reports fix failure by setting the latitude and longitude to a NaN (possibly with a failure code in the exception value) you can handle that in JS but only through some intermediate representation in JSON.

vlovich123• 6 years ago

Honestly I think it's the biggest misfeature of JSON. Arbitrary precision numbers should have been handled differently & I've repeatedly seen JSON parsers incorrectly handle those. We're now in a situation where you can't serialize numbers you encounter daily into JSON and you can't trust that numbers you correctly serialized into JSON will be properly deserialized by the peer.

I spent a lot of time writing the number handling code in PBNJSON for WebOS back in the day (IIRC there were 0 C/C++ parsers that offered correct number parsing at the time). It also makes sure to properly report overflow/underflow/failure to convert whenever you encounter situations that aren't possible (e.g. reading 32-bit number of a 64-bit int JSON, reading 64-bit integer for doubles that are out of range, etc). I'm sure I missed some corner cases though as it's insanely difficult to get every single corner case (especially with floating point).

mormegil2• 6 years ago

I've mentioned it before, but JSON Numbers are arbitrary-precision floating-point. They can't be naively stored as JavaScript numbers in all cases, since JS numbers are IEEE754 double-precision floating point.

Well, RFC 8259 says "This specification allows implementations to set limits on the range and precision of numbers accepted." and goes on to explain interoperability with IEEE doubles.

e12e2• 6 years ago

Which implementation though? If you're only talking to yourself - you don't really need to (de)serialize to/from json - in many (most) use cases the json serializer will be different from the deserializer, and the data structure will be in a different memory model and/or runtime... And I don't think there's a standard way to tag a json document hinting that 32bits is enough, or here comes a 128 bit float?

dan-robertson• 6 years ago

This is true in theory but not in practice. Most implementations don’t want to pay the cost of converting everything to bignums (or the convenience (and possibly speed) cost of leaving the numbers as strings). Generally the json-parsing is separate from the rest of the deserialisation so there’s no great way for the parser to know when it sees a number if it should convert to float or int or do something clever.

As soon as you have something in your pipeline that converts to floats, it is poisoned. Lots of implementations you may use do this conversation though (e.g. basically anything JavaScript, jq) so it is pretty easy to get float poisoning.

The actual solution to representing big bums in practice is to wrap them in double quotes. It only costs two bytes (and if you care about those bytes then you should probably not be using json.

thayne• 6 years ago

It only costs two bytes

And having to have special code to handle getting it as a string instead of a number.

jakub_g• 6 years ago

The author has many blog posts on floating point math, and they're all very interesting. If you're into the topic, you can check the tagged entries:

https://randomascii.wordpress.com/category/floating-point/

In general the whole blog is super interesting and many times submitted to HN:

https://news.ycombinator.com/from?site=randomascii.wordpress...

Spivak3• 6 years ago

I really really wish these things were errors. Data loss should require an explicit opt-in. It's easy to point to the spec and say that people should obviously know it back-and-forward but it's extremely surprising that JSON decoding is a lossy operation and you have reach not for an option flag but a whole 3rd party library to not get that behavior.

JSON's arbitrary precision decimals is the silliest footgun. When every implementation implicitly defines its own valid format by way of what native types it maps it makes for a really awkward way to pass messages between two black-box systems.

lmilcin• 6 years ago

I feel like a lot of discussion is on floating point accuracy and artifacts and most of it misses the point completely.

It doesn't help there are IEEE standards for fp calculations which are also very misunderstood. The point of these standards is to provide reproducibility and repeatability so that different types of software running on different machines or with different compilers provide same results.

The goal of floating point never was to provide precise representation or calculation and in fact it is not possible to provide in general with floating point. The goal was to make it easy and relatively fast to perform real world calculations.

You understand your calculator gives approximate results except for very special cases, so why would you bicker over some digits that are very far to the right of your result.

It is actually much worse with calculators because they tend to employ very different calculation engines and very different precision. You can actually identify a particular model or model family of calculator by running couple calculations and finding what kind of artifacts are present in the results, exactly.

Yet nobody will tell you calculators are useless as I frequently hear about floating point.

I work with Java and there seems to be this movement recently where developers use arbitrary precision arithmetic to do any sort of calculation on anything without actual idea of when this is needed. Supposedly flating point is broken and cannot be trusted. Sigh...

T-hawk• 6 years ago

You bicker over the digits far to the right because there tends to be ways for those inaccuracies to propagate up into your significant figures. That doesn't happen from typing a few arithmetic operations into a handheld calculator. It does in a computational environment that might be looping over and summing a million or a billion elements and applying more operations to them. It became too easy and fast to use floats for real world calculations, overstressing and exposing their inadequacies.

Of course, the right solution is to use the right tool for the job: algorithms that either use a better structure than a 64-bit float, or are smartly designed enough to avoid their inadequacies. But how many developers have both enough insight and enough development time to do that? Everything is a compromise; fixed-point decimal representation is one that works for many applications.

mehrdadn• 6 years ago

There's a huge difference between mathematical calculations having round-off error and computations that are not mathematical calculations exhibiting something akin to round-off error anyway.

It's like giving someone a $10k check and seeing them get $9,999.99, and then getting irritated when people try to investigate what happened because it's only a penny to you. It's kind of missing the point that the error logically shouldn't be there in this situation.

spullara• 6 years ago

And now you know why the Twitter API has all IDs as both numbers and strings.

Floating Point in the Browser, Part 1: Impossible Expectations

Related Stories