Adventures of an imperative programmer in the land of fp

http://jacquesmattheij.com/Adventures+of+an+imperative+programmer+in+the+land+of+fp

by jacquesm • 16 years ago

69 85 16 years ago

sofuture3• 16 years ago

I've recently jumped into Erlang, and have come to some of the same conclusions. I read Armstrong's Erlang book, and thought 'that seems cool' and didn't at all grok what was going on beyond the superficial. Then a few months later, just sat down and started solving Project Euler problems. At first it was strange and foreign and I was mad variables were immutable. A day later and it just clicked, I'd never had that much fun writing code.

The FP paradigm is (to me, even after 15 years of imperative programming) so much more natural for development, since you're able to sanely start attacking the problem directly -- instead of architecting a big-picture solution up front that's probably wrong anyway because you've ignored some detail you haven't yet discovered.

Building things with actions instead of objects just makes much more sense. Reuse comes much more naturally, and doesn't seem as contrived as a lot of OOP reuse seems. I've noticed that working my FP muscles out has made me a much better imperative programmer -- I write a lot more clever and effective code (not clever like 'tee-hee-no-one-will-ever-figure-this-out'!).

jacquesm11• 16 years ago

For me one of the real eye openers was that I'd gotten to the end of a fairly long book on FP without seeing a single assignment of a value to a variable.

rgoddard3• 16 years ago

I have just finished the first iteration of a web app written in clojure and in the several hundred lines of code, I have a single variable which changes state.

mattyb• 16 years ago

Which book?

WilliamLP4• 16 years ago

One thing that raises my eyebrows is how much functional programmers talk about Project Euler problems. The actual programming for solving these problems is in fact extremely trivial. They require some mathematical insight, especially after the first one hundred or so, and you need to do some external research on Pell's Equation to avoid getting stuck, and you need a fraction library if your language doesn't have it built in. But am I wrong in thinking that these kind of problems are almost no test or strain for your actual programming at all?

nudge2• 16 years ago

No mystery. Project Euler is a useful set of problems to tackle when learning a new language. It's not 'Wahoo with functional programming I can finally get past Euler 15'; it's 'Wow after a bunch of these Euler problems I think I am finally starting to get functional programming'.

sofuture3• 16 years ago

They're an absolutely fantastic way to start 'getting' any language. They're small, discrete, and varied tasks that require you to build different types of operations and structures. Sure, they're not at all reflective of 'real programming' nor are they necessarily particularly challenging, programming wise.

I'd rather cut my teeth in a new language on the first 50 or so PE problems, than take on a bigger, less defined, or more domain limited task.

I'm conversant in Python and Erlang because of PE problems entirely. They've enabled me to start actual projects in both languages.

DanielBMarkham2• 16 years ago

New to FP here as well (couple of years playing, more serious work in last few months)

It used to bug me too, but then I realized that common mathematical problems are the only "easy" way to explain how some things work.

Put another way, the problem domain in most applications is laid bare in FP. So your structures are intricately tied to whatever you're doing. You can either put up with a 30-minute "backstory" on why you're writing function foo, or we can just go with some kind of math deal. Usually authors pick the math option.

I found that once I could plow through writing short snippets, then I could start reading example code from the web. That helped a lot.

ionfish• 16 years ago

I wonder whether it's just that programmers who enjoy functional programming are also the sort of people who enjoy Project Euler problems. Since most of the problems are fairly mathematical, they also tend to have clear and concise expressions in functional languages.

davidw2• 16 years ago

I've used Erlang for a while, although sporadically. Honestly, I think I can do more with less code in Ruby, though. Erlang's great for the things it's good at, but brevity and clarity are really things I think are critical.

Here's a small example I bumped into the other day, providing a default value if a hash value isn't present:

    foo[:bar] || "beebop"

    case (catch dict:fetch(bar, Foo)) of {'EXIT', _} ->
                        "beebop";
                Val -> Val
          end

(or something close, there might be a typo)

sofuture3• 16 years ago

So wrap it in a function

  default(Dict, Key, Default) ->
      case dict:is_key(Key, Dict) of
          true -> dict:fetch(Key, Dict);
          _ -> Default
      end.

joshhart• 16 years ago

This is built into the Scala API - foo.getOrElse(bar, "beebop")

Stuff like this should be built into the standard language.

Nycto• 16 years ago

Thank you for the reference to Project Euler. I'm learning Scala right now and it gave me just the right sized challenges for testing my knowledge. They're big enough to take time and thought, but small enough that you can solve them within the CLI interpreter.

DrJokepu• 16 years ago

One thing you haven't mentioned but it is related to anonymous functions (or lambdas) and is an important part of the fp style is passing around functions as first class objects. It is quite unusual in imperative programming and it is normally only used there to provide callbacks.

Let's say, you want to write a function that exports your video library to an arbitrary medium. In FP, one way to do this is to create generator functions that create DVDs, Blu-Ray discs, etc. Then your export function would take the input and a generator function, and export the library using that function. In Common Lisp:

  (defun make-dvd () ... )
  (defun make-blu-ray () ... )
  (defun export-library (data make-medium) ... )

And then you would

  (export-library *my-data* #'make-dvd)
  (export-library *my-data* #'make-blu-ray)

Or if you want to use an inline lambda (anonymous) function:

  (export-library *my-data* (lambda () ...))

jacquesm11• 16 years ago

I'm going to do a completely separate post on the subject of first class functions because I think it is too complicated a subject (and with too many implications) to be squeezed in there as an aside, besides that I don't think I'm qualified just yet to write the article in a strong enough way yet. More understanding is required first.

It must be funny to all the FP gurus here to see someone struggling to understand the things that are second nature to them, but I find that it is surprisingly hard to teach this old dog a new trick. One part of me wants to say 'enough of this' all the time and reach for a C compiler just to get the job done :)

ww5205• 16 years ago

Javascript's treatment of anonymous function is the easiest to understand for imperative background since it uses the same syntax as a named function.

Defining an anonymous function:

    function(a, b) {
        return a+b;
    }

Giving an anonymous function a name (or assigning to a variable):

    var add = function(a, b) {
        return a+b;
    }

Defining a function with a name (just a shorthand of the above):

    function add(a, b) {
        return a+b;
    }

This makes it clear that the object stored in 'add' is a function object and can be passed around like other objects. That's the essence of the 'function as a first class object' concept.

If you have the time, I highly recommend watching the SICP video. It very gently introduces most of the FP concepts and many others. It's a classic. http://freevideolectures.com/Course/2378/Structure-and-Inter...

mquander• 16 years ago

I haven't had the chance to test it out yet, but my impression has always been that first-class functions should be presented to imperative/OO programmers as a replacement for polymorphic functions.

Frequently, for an abstraction X where I need different but similar behavior in different situations, I'll implement X by passing around functions in Scheme or Clojure, but I'd implement X with inheritance and polymorphism in C#. The frequent advantage (in my opinion) is that passing functions is more flexible, conceptually simpler, and usually shorter to write.

Do you see it that way?

jules5• 16 years ago

The real trouble here is probably closures.

    function foo(x) { return function(y) { return x+y; }; }

    bar = foo(3)
    bar(5) 
    // Returns 8, wait how did the x in the inner function remember the 3?! 
    // It's supposed to be gone already because the call to foo is over...

One way to understand this is to notice that it always "just works". It seems like magic. The 'aha' for me was when I learned how closures are implemented.

eru5• 16 years ago

I don't really get why people are so excited about anonymous functions.

First class functions may be exciting. Not having to give a name to your functions, is not.

beagle3• 16 years ago

There are about a thousand FP programmers to every imperative programmer, if not ten thousands. They use a tool you may have heard of, called "Excel", which is quite limited in scope.

It is an FP subset which is at the same time trivial to understand (much more so than imperative programming for most users!), and coupled with a usable I/O capabilities, is surprisingly sufficient for many uses.

(IDE, documentation, maintainability all suck, though; I wouldn't recommend it as your main FP tool if you can avoid it)

ww5205• 16 years ago

And many Java programmers doing FP programming unknowingly when writing an Ant script.

silentbicycle10• 16 years ago

No - ant is logic programming, not functional programming. Same with make.

bambax• 16 years ago

Yes! Excel formulas === FP! And as you say, there are orders of magnitude more Excel users than imperative programmers, so FP can't be as hard as is usually claimed, and certainly not as "unnatural".

(Before Excel 5 and VBA that Joel Spolsky claims he invented, macro language in Excel was also FP; it was a crazy but fascinating language in which I developed a whole billing application (in 1992...) I miss this language.)

Hexstream• 16 years ago

I'm a bit surprised whenever someone says that Excel is functional programming. It's more accurately described as dataflow programming.

To illustrate, Common Lisp directly supports functional programming but not dataflow programming (though it's possible to implement as a library).

Goladus• 16 years ago

Stuart Halloway has a great explanation of anonymous functions in "Programming Clojure." He identifies the specific conditions where you might choose to use an anonymous function given that, for readability reasons, naming functions is usually a good idea.

ww5205• 16 years ago

It's great that more people are exposed to the functional programming style. Kudos to OP for trying something new.

It bugs me whenever FP people talk about state being bad as if it should be avoided at all cost. State is bad if its scope is not carefully managed. Global state is generally bad because its scope allows the whole program to modify it, making it difficult to follow its validity. Local state maintained as a local variable in a function is perfectly fine. Its scope is small and its changes can be tracked easily. Pure functional code actually also implicitly maintain state in their parameters and return values, and the passing of the return values as parameters to next function.

silentbicycle10• 16 years ago

It's ultimately about making code easier to reason about. Immutability (at least as a default) makes the dataflow between independent portions of your program clearer, since every value is determined by where it came from, not where it came from and everything that could have potentially touched it along the way.

silentbicycle10• 16 years ago

Oh: Easier for the compiler to reason about, too. Not just the programmers.

blintson• 16 years ago

Towards the end of his entry he mentioned something about readability. I believe postfix notation is superior in both read & write-ability for FP.

Take: (reduce (lambda (x y) ...) (map (lambda (x) ...) data-set))

When you actually read this what do you do? You work inside-out to understand it. You figure out what 'data-set' is, then you figure out what '(lambda (x) ...) does to it, and so on.

You (or me at least) also write the code inside-out. You start with the data, and an idea of how to transform it, and you work your way towards that transformation.

Compare to:

((data-set (lambda (x)...) map) (lambda (x y) ...) reduce)

Of course, this brings up a lot of edge-cases. Ex.: Where does 'define' fit into this? You really want define and the variable name at the beginning.

jules5• 16 years ago

This, in my opinion, is one of the greatest strengths that OOP has over FP: data.map(...).reduce(...). It doesn't really have anything to do with OOP, you could just as well have such a syntax for calling functions. In F# you do have the |> operator that does something like this.

This may seem superficial, but it helps readability a lot. The human mind (or mine at least) is just not well suited to unraveling nested structures.

koenigdavidmj2• 16 years ago

Whoa! The UNIX shell's pipelines!

jimbokun2• 16 years ago

You can get a similar effect in Clojure with the "->" operator:

http://clojure.github.com/clojure/clojure.core-api.html#cloj...;

So you can take

    (foo (bar (baz "Hi!")))

and make it

    (-> "Hi!" baz bar foo)

euccastro• 16 years ago

I'm writing from a phone and lost a longer reply to expired link. In short:

You and the OP (cf. his assumption that you read functional code inside out) seem biased towards a chronological, bottom up reading of code.

An outside-in reading of prefix code is useful to get a top-down general grasp of the structure.

Both approaches are useful and complementary.

This duality also applies when writing code. Cf. "wishful thinking" in SICP: sometimes you want to assume away auxiliar or extraneous functionality to sketch an outline of your program.

In both reading and writing, alternating approaches helps to find the kernel of the problem quickly and to build a whole understanding at your pace, so you don't get bored or stuck.

To that effect, I find prefix notation more balanced, that is, easier to read both ways.

As a practical note, if you follow the 80 column convention long().call().chains() are space hogs and awful to break up. The equivalent problem in lisp is solved with a two-space indent.

But that's just an instance of the syntactic sugar debate. My point is not about notation but how you approach the code.

DanielBMarkham2• 16 years ago

Welcome to the party, Jacques! I think you and I started about the same time.

FP seems much richer than IP, but maybe that's just me. I know that once you get all the basic set operations, then you move on to continuations and Monads and it's like wow! A whole other world opens up. Then you can move on to all sorts of other cool stuff like super-scaling which kind of just "falls out" of FP. So it seems like there is more depth here for geekiness.

As far as bugs, I guess that the vast majority of bugs are related to either "state leakage" -- somebody tickling your variables while you're not looking -- or off-by-one errors. FP eliminates both of those. I know I try to stay as immutable as possible and my code feels a lot more solid than it used to.

bufo• 16 years ago

it's funny, in French engineering schools we have the opposite reaction; in "prep" schools students are taught OCaml which is their first programming language if they're not geeks.

Functions returning functions seem a natural thing as they are used to the exact same kind of abstraction in math (and even sometimes order 3+ functions when you study duality!). Conversely, they are initially puzzled when they are taught Java in their engineering school because of the difference between static variables and attributes, constructors and other unnatural concepts.

eru5• 16 years ago

`Unnatural' in the sense of `not relating to natural transformations'?

Dn_Ab3• 16 years ago

Unlikely since natural transformations are a statement on how a particular collection of functions behave under composition.

Dn_Ab3• 16 years ago

yea, anyone who has ever done differentiation or messed with probabilities involving random variables has played with 'higher order functions'.

lolipop12• 16 years ago

In some schools, you actually start with lambda calculus, just a bit more theoritical.

groaner• 16 years ago

Maybe it's a national pride thing as well, since OCaml was developed at INRIA.

chriseidhof• 16 years ago

If you really start studying FP literature you'll discover that an enormous amount of ideas that are in vogue in today's languages have been invented years ago by functional programmers. For example, this paper from 1965(!) describes DSLs: http://www.cs.cmu.edu/~crary/819-f09/Landin66.pdf

I think learning a language like Haskell can be extremely good for you as a programmer. The problem is that you just can't expect to be productive, if you're new to it, and that might be very frustrating if you're doing to do work. However, just as jacquesm writes, if you're doing it for fun you'll learn a lot (that you can sometimes apply directly to your normal programming).

eru5• 16 years ago

Interesting read.

From the article: "You have to train yourself to start the understanding of code you're looking at from the innermost expressions, which are the first to be evaluated."

The author is in for some even more mind bending, when he eventually has a look at lazy languages.

jacquesm11• 16 years ago

Lazy evaluation: evaluate an expression only when you are actually going to do something with the result. For instance, in a generator you can generate 'infinitely long lists' because only those values that are consumed are actually generated.

Is that what you mean ?

Or do you mean when that principle is expanded to the whole language ?

eru5• 16 years ago

To the whole language. Then you can't tell what's evaluated when, easily.

martingordon• 16 years ago

I tried wrapping my head around Clojure but I just couldn't. I'm currently diving into Scala and finding it much easier to get into coming from an OOP background (I'm a Java developer by day and a Rails/Objective-C developer by night).

Adventures of an imperative programmer in the land of fp

Related Stories