The trouble with 'floor' and 'ceil'

http://mortoray.com/2016/01/05/the-trouble-with-floor-and-ceil/

by nikbackm • 11 years ago

14 14 11 years ago

joekinley• 11 years ago

What is "unexpected" about floor(17.999999) to return 17 instead of 18?

Maybe my assumptions are wrong, and this is a truly curious question. I would ALWAYS expect floor to "basically" return the number in front of the ".", however close it is to another number. If you want that, you should use round().

mortoray5• 11 years ago

This is the "correct" behaviour of `floor` obviously, but the unexpected part has to do with floating point accuracy and use domains.

Numbers will end up slightly lower or higher than an expected value simply due to accuracy in calculations and minor differences in input.

For example, suppose we have two UI elements, one with a width of 80% and the other expressed as a factor as 0.8. Depending on how specifically these are parsed and processed this may result in slightly different values. For the user creating the UI however they'd expect the same result.

The epsilons I introduce into my floor/ceil code is meant to deal with these minor variations.

mchahn• 11 years ago

Why not round everywhere? Wouldn't this resolve the issue?

davvid• 11 years ago

I think another contributor to the author's surprise is this subtle detail about the behavior of Python's `print` statement. From the docs[1]:

Python only prints a decimal approximation to the true decimal value of the binary approximation stored by the machine. If Python were to print the true decimal value of the binary approximation stored for 0.1, it would have to display

    >>> 0.1
    0.1000000000000000055511151231257827021181583404541015625

That is more digits than most people find useful, so Python keeps the number of digits manageable by displaying a rounded value instead

    >>> 0.1
    0.1

[1] https://docs.python.org/2/tutorial/floatingpoint.html

hibbelig• 11 years ago

Maybe you need rational numbers instead of floating point numbers. Then .8 and 80/100 can be equal.

santaclaus• 11 years ago

I would ALWAYS expect floor to "basically" return the number in front of the "."

Not for negative numbers -- floor(-0.1) should be -1.

tasty_freeze• 11 years ago

The author of that piece hasn't thought hard enough about the issues. floating point, no matter if it is binary, decimal, base-100, or oct-precision will always cases where numbers are not exactly precise. That is an inherent issue with floating point that must be considered, and the author makes a mistake to attributing his problems to floor() and ceil() instead of floating point.

0.005 works for his domain, and someone else may find that 0.001 is what is right for them, and the existing floor() and ceil() functions give them predictable tools to do that. If he only needs to draw boxes on integer aligned grids, that solution is fine, but it will break all over again if he tries drawing rotated boxes, lines, triangles, etc (plus he'll find other numerical issues that he can currently ignore).

He says 0.005 is an acceptable fuzziness. But now 17.9949999 will round down and 17.9950000 will round up. That cliff is still there; he has just moved it and thinks he solved it. A guy I used to work with called this "squeezing a balloon" -- you didn't fix the bulge, you just moved it somewhere else.

Finally, he appears to approve that printf prints 18.0 when the input was 17.999999. That is convenient in some cases, but a lie in others:

a = 17.999999 b = 18.0 print a, b --> 18.0, 18.0 print a - b --> 0.0000001

mortoray5• 11 years ago

I agree my initial examples were not clear enough to illustrate the problem precisely. I've added a section now about idempotance and how this slight varation in floating point is causing my original problem.

Your point about floating point not having exact values is precisely the issue, and that is what my added example tries to demonstrate. My previous article, linked from the intro, about floating point tries to cover that aspect a bit more.

http://mortoray.com/2015/07/06/essential-facts-about-floatin...

donatj• 11 years ago

I had some very similar, very troubling PHP code.

  var_export($var) // 18
  json_encode($var) // 18
  serialize($var) // 18i:18;
  var_export($var == 18) // false
  var_export(floor($var)) // 17

Ends up PHP floats have a configurable precision and an actual physical 32/62 bit (depending on the system) in memory value who are separate and some operations use one and some the other. It was particularly troubling because I'd hope to be able to serialize and unserialize numbers losslessly but that was not the case.

lsaferite• 11 years ago

Can you provide a working 3v4l.org example of this issue? I'm very interested is what triggered it to not output the correct values for json_encode and serialize.

yoklov• 11 years ago

This is just sweeping the problem under the rug. Round would probably be a better choice for them, but it wouldn't solve the problem, since you can't get rid of that discontinuity.

Generally a good rule of thumb is that unless you are certain you need floor or ceil, use round.

mortoray5• 11 years ago

Yes, round is usually a safer choice, but in this case we had reasons to use floor and ceil, and thus had to deal with the issue.

I've added a section to the article with a more correct snapping function showing the issue more clearly (I hope).

The trouble with 'floor' and 'ceil'

Related Stories