On calculating Fibonacci numbers in C

http://blog.noblemail.ca/2013/01/on-calculating-fibonacci-numbers-in-c.html

by avibryant • 13 years ago

109 54 13 years ago

blog.noblemail.ca

protomyth6• 13 years ago

"....since the 96th Fibonacci number is the largest to fit in an unsigned long int"

Have to agree with the first person to comment on the article's site. If we are going for speed, just pre-calc the 96 numbers and do an array lookup in the function guarded by and if.

tlarkworthy• 13 years ago

and only perform any optimizations at all after you have proved its a bottleneck for your application (unlikely) :p

protomyth6• 13 years ago

I just thought it was funny the article took a shortcut on speed testing because only 96 values were possible from the function. At that point, we are no longer talking about math or math error, we are talking a simple array lookup.

I get the "less math" part of the article, but it just seemed a bit confused.

orionblastar3• 13 years ago

This is basically a table lookup. You could do the same for any series of numbers like the prime numbers.

You could store them in a database table even and then pull out the result by a key with the number assigned to it that represents the Nth Fibonacci number. It might even be faster than an array loaded from a text file, and if you happen to have limited memory and cannot store the entire array in memory some how later on when you use a different data type for large numbers and go past 96 numbers, the database method may even be faster and use less memory. You might even have to store the big numbers as a blob or text or something if the database cannot handle numbers that big.

protomyth6• 13 years ago

If I don't have room to store 96 numbers in an array in the function, I am going to have problems elsewhere.

snoble6• 13 years ago

Sure, but it's not a post about the bestest solution. It's a post about how using more advanced math can ("can" not "will") lead to a worse solution for your context. Fibonacci numbers in a long int is certainly a toy example; but it's a toy example that had an actual reference.

just2n3• 13 years ago

The article really isn't about speed, it's more that engineers shouldn't necessarily take mathematical results as the end-all to their considerations when developing an algorithm. Math is important to formulating algorithms, since it can provide, in theory, a closed form solution for something typically done iteratively (the example here: fibonacci numbers). But you still need to verify when an algorithm is going to be correct. In this case, it's pretty well known how inaccurate floats and doubles can be within X significant figures, and this is why I've always said the analytical solution doesn't really work on a computer unless you have a CAS library that can approximate an exact answer to an arbitrary number of significant figures (and it still won't be accurate if shoved into a float/double afterwards). So there's a trade-off, and choosing the theoretically best solution (a closed form solution) isn't necessarily the best possible choice for the task. In this case, yes, if we only need the first 96 fibonacci numbers, ever, in our program, it makes more sense to pre-compute them. However, if we only do the computation once in the entire program, even that doesn't necessarily make sense. It entirely depends on the application, but that's the entire point: a closed form for fibonacci numbers isn't the end of considerations.

I am curious though, why is it so many people are oblivious that there's an O(logN) time algorithm to compute fibonacci numbers iteratively on integers? It's a beautiful piece of math, and very simple so everyone should understand it. Just notice that:

    ((1, 1), (1, 0))^N -> ((f_{N+1}, f_N), (f_N, f_{N-1}))

given N > 0, where f_N denotes the Nth fibonacci number. For SW engineers who are actively interviewing: does anyone still ask you to write a function to compute the Nth fibonacci number in technical interviews?

snoble6• 13 years ago

I'd greatly appreciate it if you left this as a comment on the original post.

protomyth6• 13 years ago

I get the point of the article, but it seems like defaulting to a compiler optimization that might not be in other compilers is a poorer solution than examining the problem for another Math solution. You yourself have shown one.

Computer precision is a problem and programmers should learn a lot about how ints and fp work, but throwing out all Mathematical solutions is really not a good idea.

orionblastar3• 13 years ago

I used to use the two bucket method to generate a Fibonacci number, thanks for showing me another way to do it.

I am working on Ackermann Numbers now, but it takes a very long time to generate them and I have to either write my own classes to handle big numbers or using a different library to use large numbers. I moved from C to C++ to do that. Your unsigned long int variable would run out of room after the 96th Fibonacci number, and you might have to move to C++ and use a custom class as well to be able to handle those big numbers.

dalke• 13 years ago

Note that the author did write "since the 96th Fibonacci number is the largest to fit in an unsigned long int".

jdreaver2• 13 years ago

I still don't understand how the author jumped from a cute comparison of Fibonacci algorithms to saying programmers shouldn't learn more math. Math isn't just arithmetic and numbers. He acts as if the recursive solution isn't also math.

Developing mathematical maturity is, in my opinion, an excellent way to think more clearly, precisely, and rationally about any problem, even if it doesn't necessarily involve numbers. The author could make the case that number theory in particular isn't directly applicable to many problems (I do have a lot of fun with Project Euler though), but again, math isn't just about numbers. Math teaches you how to reason about the relationships and patterns between objects (which can be numbers, vectors, spaces, groups, rings, etc. In a certain sense, you become more comfortable with higher levels abstraction, and it requires you to be precise and explicit about assumptions.

That's jut my two cents. Math education is pretty dismal in the US, but math can be very enjoyable if you find the right textbook, teacher, or even free online class nowadays. Never fear math.

EDIT: The author has pointed out that he is making a play on words between "math" and "Math" (the mathematical library). I didn't understand that when I wrote this post.

snoble6• 13 years ago

I do distinguish between math and Math. Decide for yourself what that distinction means.

mikeash• 13 years ago

The shift key?

jdreaver2• 13 years ago

I reread the article with that in mind, and it makes much more sense, to the point that I am inclined to agree with much of what you said now. I think you should highlight that difference closer to the beginning of the article. That is a nice, subtle play on words.

Strilanc• 13 years ago

The analytic solution to the Fibonacci sequence is nice and all, but I generally dislike methods that require you to very carefully check how much precision you need. I much prefer the matrix exponentiation by repeated squaring approach [1].

1: http://ronzii.wordpress.com/2011/07/09/using-matrix-exponent...

mathattack• 13 years ago

I came to the exact opposite conclusion as the author. Programmers should know a lot of things, including both recursion and math since tere are many ways to approach a problem.

tyler-dodge• 13 years ago

Wait Im confused. Wouldnt the point of using the analytic approach be that i could start at any n besides 0? Because of course if im just taking the sums from 0 to n it'd be faster to do it the tail recursive way because thats just simple addition.

dubya3• 13 years ago

The post didn't go into this, but the sum has a lovely closed form as well. The sum of the (phi^n-psi^n) terms is the difference of two geometric series, so you can apply sum(r^i, i=0..n)=(r^(n+1)-1)/(r-1) to get a formula that evaluates the sum exactly.

xyzzyz• 13 years ago

It's interesting to note that one can get O(lg (n)) complexity of analytical solution without using any floating arithmetic. There are at least two easy ways to do it: one is to recognize the analytical formula as a result of diagonalizing certain linear map, and calculate nth power of that map using repeated squaring instead of diagonalization. In more concrete terms: consider a square matrix A = {{1,1},{1,0}}. To obtain nth Fibonacci number, raise A to the nth power and take lower left entry. Clever multiplication by repeated squaring lets one perform only a logarithmic number of matrix multiplications instead of linear number using naive method.

Another way is to interpret analytical formula in Q(sqrt(5)) ring, and implement Q(sqrt(5)) arithmetic using quadruples of integers. More concretely: suppose we have two numbers of the form a+b sqrt(5), c+d sqrt(5), where a,b,c,d are rational. then their sum is (a+c) + (b+d)sqrt(5), and their product is (ac+5bd)+(ad+bc)sqrt(5). Thus, just like we usually represent complex numbers as pairs of real numbers with special multiplication formula, we can represent elements of Q(sqrt(5)) as pairs of rationals with special multiplication formula. then we just use the analytical formula for our special number system. I've implemented it a while ago in Haskell: https://github.com/xyzzyz/FibBinet

stephencanon3• 13 years ago

While I agree with the gist of your comment, it's important to note that none of these give an O(lg(n)) algorithm for computing fibonacci numbers (except on a hypothetical computer that can do arbitrary size integer operations in O(1) time).

The nth fibonacci number has O(n) digits, so you can't even write it out in time less than O(n). The repeated-squaring approach will involve multiplying numbers with O(n/2) = O(n) digits in its final step, so a very coarse analysis says the complexity is at least O(n lg n).

rcfox• 13 years ago

You're conflating the magnitudes. You can't directly compare the number of multiplications with the number of transistor switches or CPU cycles required for a single multiplication.

Also, the number n has O(log(n)) digits.

blackhole• 13 years ago

This is completely missing the point of Big-Oh notation. If you start trying to analyze how long integer operations take, ALL your computational complexity analysis falls apart for any sufficiently large integer arithmetic for any algorithm. This is precisely why such operations are ignored, because it becomes completely impossible to do any meaningful platform-independent analysis if you try to take that into account.

That's not to say you shouldn't take it into account, but the algorithm he described is, in fact, O(log(n)) time. It just is. That's how the math works. Now, you should note that the O(log(n)) time constraint does not take into account arbitrary integer size which will cause progressively larger constant factors to influence the efficiency of the algorithm... but it is still a O(log(n)) algorithm.

iansimon• 13 years ago

The algorithm is also relevant if you want to compute the nth Fibonacci number mod k.

dubya3• 13 years ago

That's how I would approach this if arbitarily large integers were needed. It's particularly nice because it readily generalizes to any other finite depth linear recurrence.

The analytic solution really should win for very large terms though, but I'm not sure how to implement it properly. Can someone outline how you would decide what precision is needed to compute the Nth Fibonacci number?

gizmo686• 13 years ago

The nth term of the fibbonochi sequance can be written in the form (A^n-B^n)/sqrt(5) [1], where A=(1+sqrt(5)/2) and B=(1-sqrt(5)/2)

Lets say that A` and B` are approximations of A and B, that are within E of the actual answer, that is to say |A`-A|<E and |B`-B|<E.

The total error can be found by:

|(A^n-B^n)/sqrt(5) - (A`^n-B`^n)/sqrt(5)|

If we can make this error be less than 1/2, we can simply round to the nearest integer and have the correct answer.

|(A^n-B^n)/sqrt(5) - (A`^n-B`^n)/sqrt(5)| < 1/2

|(A^n - A`^n + B`^n - B^n)| < sqrt(5)/2

If both of those absolute values are less than sqrt(5)/4, than their sum would be less than sqrt(5)/2, satisfying the inequality. Because 1<sqrt(5/4), I will make them be less than 1. Without loss of generality, I will consider only one of the absolute values.

|A`^n - A^n| < 1

Assuming the maximum error:

|(A+E)^n - A^n| < 1

Notice that the first term when you expand the binomial is A^n, so the n-1 remaining terms are our error. Every coefficient is less than 2^n. The greatest that A factor can be is A^(n-1)<A^n. And the greatest that the E factor can be is E^1. Therefore all (n-1) factors are less than (2^n)(A^n)E.

The sum of these error term is therefore less than (n-1)(2^n)(A^n)E. So:

(n-1)(2^n)(A^n)E < 1

Because A<2, A^n<2^n we can say: (n-1)2(2^n)E < 1

E < 1/(2(n-1)(2^n))

If our value for A is accurate for d+1 (binary) digits passed the decimal point, E<=2^(-d).

2^(-d) < 1/(2(n-1)(2^n))

2^d > 2(n-1)(2^n)

d>lg(2)+lg(n-1)+lg(2^n)

d>1+lg(n-1)+n

d>2n

I made a lot of simplifications in this analysis, so you can probably get away with fewer terms. It is interesting to note that Fibonacci numbers grow exponentially, so the number of digits needed simply to right the answer grows linearly, so the extra digits do not decrease the algorithmic efficiency.

[1]http://en.wikipedia.org/wiki/Fibonacci_number#Closed-form_ex...

orlandu632• 13 years ago

In the same vein, there's also an O(log(fib(n))) recursive algorithm:

    fib :: Integer -> Integer
    fib 0 = 0
    fib 1 = 1
    fib 2 = 1
    fib n = case n `quotRem` 2 of
        -- if n = 2*m
        (m,0) -> (fib (m+1))^2 - (fib (m-1))^2

        -- if n = 2*m + 1
        (m,1) -> (fib (m+1))^2 + (fib (m  ))^2

shrughes• 13 years ago

Why not just say O(n)?

gus_massa• 13 years ago

If you "drop" the 1/2, you can use Z(sqrt(5)) and work with only a pair of integers. Also, the two parts of the formula are "conjugates" so it's possible to get the exact result with only one half.

Define: Im5(a+b * sqrt(5)) =b

Fib(n)= Im5 ((1+1* sqrt(5))^n) / 2^(n-1)

just2n3• 13 years ago

Re-commenting on OP @ author's request:

Math is important to formulating algorithms, since it can provide, in theory, a closed form solution for something typically done iteratively (the example here: fibonacci numbers). But you still need to verify when an algorithm is going to be correct, and more importantly, appropriate. In this case, it's pretty well known how inaccurate floats and doubles can be within X significant figures, and this is why I've always said the analytical solution doesn't really work on a computer unless you have a CAS library that can approximate an exact answer to an arbitrary number of significant figures (and it still won't be accurate if shoved into a float/double afterwards). So there's a trade-off, and choosing the theoretically best solution (a closed form solution) isn't necessarily the best possible choice for the task. In this case, yes, as others have pointed out, if we only need the first 96 fibonacci numbers, ever, it makes more sense to pre-compute them. However, if we only do the computation once in the entire program, even that doesn't necessarily make sense. It entirely depends on the application, but that's the entire point: a closed form for fibonacci numbers isn't the end of considerations.

    ((1, 1), (1, 0))^N -> ((f_{N+1}, f_N), (f_N, f_{N-1}))

dbaupp• 13 years ago

(The author probably meant on the actual blog post, rather than at the top level on HN. :) )

just2n3• 13 years ago

If that's the case, I apologize, but I feel like a victim of ambiguous terminology!

aeon103• 13 years ago

If im not mistaken the iterative solution of using a simple for loop would definitely be insanely faster than a recursive algorithm which is exponential.. What am i missing here?

snoble6• 13 years ago

fork the code and post the benchmark results

dmlorenzetti• 13 years ago

The recursive approach isn't necessarily worse, provided the compiler can do a so-called "tail call optimization" (in the original posting, note the line that says of the compiler optimization level, "We need -O2 so gcc will recognize the tail-call.").

Tail-call optimization says that if a function directly returns the result of calling another function, then you don't need to add to the call stack in order to effect the function call-- you just modify the current stack frame as needed. For a recursive function, apparently a good compiler can essentially transform this into code very similar to what you'd get with an explicit loop.

None of this means that a simple for-loop wouldn't be appropriate, or even faster. Personally I use for-loops a lot, and, like you, I would naturally incline to writing this using a loop rather than recursion.

I see yours is a new account, so maybe this is just a cultural thing that I've got used to. To generalize grossly, recursion is more fashionable on HN than iteration. Every once in a while you'll see a post here that touches on iteration-- like maybe somebody will mention the tradeoffs between counting down and counting up a loop variable. But guaranteed at least once a week you'll see something on recursion and functional programming.

My impression is that most people here understand that a simple for-loop will get the job done, but they don't see for-loops as very interesting, and don't figure it adds anything to the discussion to bring it up.

Meanwhile, there are people here who will actively advocate against using for-loops. There's a whole theory of functional languages that says that if your program doesn't directly modify variables, then it's easier to reason about. A for-loop modifies the trip variable, so it's not a functional construct.

aeon103• 13 years ago

Yes, I understand that recursion is much more appealing and leads to much cleaner and elegant code than iterative solutions. and thank you for the introduction to the community I suppose.

Anyway I forked the original code, added an iterative version based on the comment by (Nick Craig Wood) (https://github.com/AeonAxan/benchmarking_fibonacci)

Here are the benchmarks [recursive fib: 7,765 ms] | [iterative fib: 8,047 ms] | [analytic opt fib: 54,875] | [analytic fib: 103,937 ms] |

And it appears that the recursive version is faster than the naive iterative version by a few hundred milliseconds.

algorias• 13 years ago

Tail call optimization does nothing to mitigate the fact that it takes an exponential number of recursive calls to calculate fibonacci. Just draw the call tree of f(5) by hand to see what happens.

>My impression is that most people here understand that a simple for-loop will get the job done, but they don't see for-loops as very interesting

Well, let's make them a bit more interesting then... The simple for loop is actually an example of dynamic programming, i.e. building a table of values from the bottom up in order to avoid repeated calculations. In this case the table only needs to have size 2, as an entry won't be read ever again after the first 2 uses.

lmkg• 13 years ago

Take a look at the recursive code being used, it's not exactly what you think it is. The naive recursive solution makes two recursive calls to the Fib function, and requires an exponential number of calls. The 'standard' recursive solution defines a helper function that takes two arguments, so each recursive step only takes one function call and the total number of calls is linear.

In general, a recursive solution need never have a bigger big-O than an iterative solution. You just need to define a helper function that has a many input parameters as your for-loop had variables that it updates.

alexkus• 13 years ago

GMP (arbitrary precision library) implements it as a lookup table for "low" n but uses a different set of identities for larger n.

http://gmplib.org/manual/Fibonacci-Numbers-Algorithm.html

dxbydt• 13 years ago

The author very casually takes a potshot at issue #41, calling it a "really silly time wasting debate". Was #41 honestly such a silly time waster? Pray, what would you have done with your precious time instead? I for one took away a lot from #41. It was a very valuable learning experience. #41 was definitely way more valuable than computing fibonaccis, if I may say so.

pmb• 13 years ago

Only the first 90ish fibonacci numbers fit in a 4 byte int. Use a lookup table, make it O(1).

On calculating Fibonacci numbers in C

Related Stories