Google Calc floating point error: 16384.69-16384.58=0.10999999999

https://www.google.com/search?q=16384.69-16384.58

by nathanwallace • 10 years ago

1 4 10 years ago

The Google Calculator returns incorrect results for the range:

  16384.69 - 16384.58 (0.10999999999)
  16385.69 - 16385.58 (0.10999999999)
  ...
  32766.69 - 32766.58 (0.10999999999)
  32767.69 - 32767.58 (0.10999999999)

Results are correct on either side of this range:

  16383.69 - 16383.58 (0.11)
  32768.69 - 32768.58 (0.11)

Piskvorrr2• 10 years ago

You say "incorrect", I say "incompatible assumptions". http://blog.reverberate.org/2014/09/what-every-computer-prog...

(TL;DR: Floating point arithmetic on binary numbers doesn't translate into decimal numbers intuitively, "weird" rounding errors appear.)

brudgers• 10 years ago

What is interesting is that, given its massive scale, Google is not performing decimal arithmetic for users. It's not that I don't understand why it isn't, but I think it highlights Google's values in regard to meeting the needs of ordinary people: e.g. how is a school teacher supposed to explain this to third graders?

Google Calc floating point error: 16384.69-16384.58=0.10999999999

Related Stories