Ask HN: How do you explain closures?

https://news.ycombinator.com/item?id=926077

by dstorrs • 17 years ago

69 64 17 years ago

news.ycombinator.com

In the past year or two, I've had to explain closures--what they are and why they are interesting--several times. Usually, at the end of my explanation, the person doesn't get it, and they go back to doing whatever they are familiar with.

So, I ask HN to put yourselves in this scenario and tell me what your answer would be:

Joe is a talented but inexperienced developer. He's got a year or two of experience, maybe as much as 5, but he programs for a job, not for fun. Also, for whatever reason he's never encountered closures--most likely he's only worked in languages that don't really support them.

You want to explain to him (a) what they are and (b) why he would want to use them. You have about 10 seconds to get his interest before he starts thinking about his WoW raid tonight, and about 5 minutes for the whole explanation.

What do you say?

shadytrees• 17 years ago

An interesting demonstration of why closures are useful is to start from a hypothetical language that supports closures but not objects and then implement a (simple!) object/class system. For me, it was extremely clarifying to have something in my mind click and go "Oh, OOP doesn't need to be a language feature ... there's this much more powerful concept available as a building block."

slmbrhrt2• 17 years ago

The venerable master Qc Na was walking with his student, Anton. Hoping to prompt the master into a discussion, Anton said "Master, I have heard that objects are a very good thing - is this true?" Qc Na looked pityingly at his student and replied, "Foolish pupil - objects are merely a poor man's closures."

Chastised, Anton took his leave from his master and returned to his cell, intent on studying closures. He carefully read the entire "Lambda: The Ultimate..." series of papers and its cousins, and implemented a small Scheme interpreter with a closure-based object system. He learned much, and looked forward to informing his master of his progress.

On his next walk with Qc Na, Anton attempted to impress his master by saying "Master, I have diligently studied the matter, and now understand that objects are truly a poor man's closures." Qc Na responded by hitting Anton with his stick, saying "When will you learn? Closures are a poor man's object." At that moment, Anton became enlightened.

jriddycuz• 17 years ago

You have much to teach us, great one.

jamesbritt• 17 years ago

No link to the source?

http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/m...

tamas• 17 years ago

The idea is really nice, but I'm not sure it would work on the OP's Joe, who probably wouldn't even care about the hypothetical language and would regard the whole concept hypothetical, only for those who have way too much free time. (With WoW on the table, you don't have that luxury.)

I would take the route of first order functions -> anonymous functions -> closures, but I think it is really a lost cause for the typical 9-5 programmers.

gruseom2• 17 years ago

I would take the route of first order functions -> anonymous functions -> closures

But those are three completely different concepts. A closure need not be anonymous or even first-order.

gcv• 17 years ago

Yes. This approach works. I used it when I actually did work in a proprietary language which had no usable objects, but did have half-decent lambdas. Most people did not understand or use them at all, but one new guy with a Java background asked me how to cope. I explained to him how to use closures to implement a simple bank account class, and he got it immediately.

I don't think this guy played WoW, though. :)

wingo• 17 years ago

You still want applicable objects, though, and closures don't quite cut it. (I say this as a schemer.)

See http://www.r6rs.org/formal-comments/comment-6.txt, for example.

chrismear2• 17 years ago

I've found that the concept of anonymous functions is the biggest hurdle to get over -- closures seem to follow as a natural extension of that.

I explained this to someone recently in the context of event handlers in ActionScript/JavaScript, starting with the idea that when you do:

  something.onclick = function(){...};

you're passing a function around just like it was any other value.

Then I moved to an artificial example of a 'magic function-making machine':

  function giveMeAnAdder(baseNumber) {
    return function (i) {
      return baseNumber + i;
    };
  }

with a couple of examples of storing and using the functions that giveMeAnAdder returns.

Then I pointed out that, normally, local variables in a function disappear when the function returns, so how does my adder 'remember' what baseNumber is? And the answer is that the anonymous function is actually a bundle of a function body and the local context, and we call this a closure.

dstorrs• 17 years ago

Thanks, this looks like a good way to present it. Next time I'm in this position, I'll use this as my jumping off point for the discussion.

mikepurvis• 17 years ago

This is my approach as well. JavaScript provides a very natural syntax for it, and the use case of "make me an event handler with some stuff baked into it" is extremely obvious and prevalent.

cridal• 17 years ago

Can people please stop using the "make adder" examples? Can't you come up with something original and fresh? The triviality of what is acomplished in such case actually obscures the usefulness and power of the concept. As a result, the novice will interpret the end product as some useless, magical mambo jumbo that doesn't give him much. So who cares if I can get a function that adds a number to something. I have a "+" for that. Give me a use case that I can admire...

Why do you want me to appreciate a ferrari if all you do with it is go pick up some groceries...

BigZaphod3• 17 years ago

That's why I use a counter as a simple demo - you see state changing with each call and it's done without using a global or passing in some kind of context object. It's not a lot more powerful than the adder, but in my experience it more often generates a "what the hell..." kind of reaction to the uninitiated.

chrismear2• 17 years ago

I agree that for a novice a trivial example doesn't inspire them to learn more. But an experienced programmer who just wasn't familiar with this concept might appreciate more a stripped-down example like this that demonstrates the essentials of the idea. And in general, different people have different learning styles.

scott_s• 17 years ago

"Original and fresh" is generally not conducive to understanding. People are more likely to understand a new concept when that new concept is applied to concepts they already understand. It lets them filter out everything but that concept.

BigZaphod3• 17 years ago

I usually go with a simple counter-generator as a first example:

  function counterMaker(next) {
    var value = next;
    return function() { value++; return value }
  }

  var myCounter = counterMaker(42);
  alert(myCounter());
  alert(myCounter());
  alert(myCounter());

I'm not sure it is necessary to explain what they are/how they work right at the start. An example like this is pretty clear that something "weird" is going on and if "Joe" is interested at all, questions should probably come up which can then be answered with more details. I think the "what to use them for" stuff is going to depend quite heavily on the environment, language, libraries, and product you're working on.

I usually find it's better to introduce a language idea to someone and let them try to find out where they might put it on their own - they may surprise you. If you're doing code reviews, you could use that as an opportunity to point out how something like a closure (or any other construct) might have improved a situation, or, at least, how it may have changed it.

btilly3• 17 years ago

I have been in this position, and what I've found is that formal definitions don't make sense. You have to encounter an example that makes you sit up and realize that something is going on that you don't know.

Two examples where I tried to induce that kind of brain lock in Perl programmers are http://www.perlmonks.org/index.pl?node_id=34786 and http://www.perlmonks.org/index.pl?node_id=50132.

gruseom2• 17 years ago

Closures are functions with state.

Your claim that Joe is a talented programmer isn't very persuasive. Edit: maybe I should say why. I have known many programmers of the 9-5 type who do a passable job, but are not interested in learning anything. It is as if they had a limited supply of energy for learning and used it up when they were getting started. This is a common type of working programmer and you made it sound like Joe might be one.

pdrummond• 17 years ago

http://www.iode.co.uk/blog/pdrummond/i_finally_understand_cl...

Tichy• 17 years ago

A method remembers its context? I am not sure how to not understand it. Understanding all the implications might be another problem.

rabidsnail2• 17 years ago

Closures are essentially a way to create private variables.

For example, in Javascript:

    var fn = null;
    
    (function(){
        
        var temporary = 789;
        alert(temporary);
        
        var secret_private = 5;
        fn = function(){
            alert(secret_private);
        };
    
    })();
    
    fn();

In javascript,

    (function(){ ... })();

defines and executes kamikaze function. It's a function which dies as soon as its job is completed.

We can use the kamikaze to hide secret_private from all but the functions that we want to have access to it.

Since fn is defined in the kamikaze function, fn can access secret_private.

Since fn is assigned to a global variable, it can be accessed from outside the kamikaze.

When the kamikaze dies, it takes all of its variables with it, except for the ones that are referred to outside of the kamikaze.

temporary disappears because it is referred to only inside the kamikaze.

But, since secret_private is referred to in fn, and fn is a global, secret_private does not disappear, but now only fn can refer to it.

So sceret_private is essentially a private variable of fn.

BigZaphod3• 17 years ago

Kamikaze functions? Sounds dangerous...

rabidsnail2• 17 years ago

Dangerous, maybe, but high affect.

slmbrhrt2• 17 years ago

I'm referring to all my anonymous functions from now on as kamikaze functions. Brilliant.

rleisti2• 17 years ago

You could try an example showing what a closure does 'under the hood' by showing an equivalent non-closure example, like:

With Closure:

  foreach (var obj in objects) {
    AddButton("Delete " + obj.ToString(), () => obj.Delete());
  }

Without closure:

  class DeleteHandler {
    public DeleteHandler(TYPE obj) {
      _obj = obj;
    }
    private TYPE obj;
    public void DoDelete() {
      obj.delete();
    }
  }
  ...
  foreach(var obj in objects) {
    var deleteHandler = new DeleteHandler(obj);
    AddButton("Delete " + obj.ToString(), deleteHandler.DoDelete);
  }

tome2• 17 years ago

I'm not convinced by the utility of your example. What's wrong with:

    foreach (var obj in objects) {
      AddButton("Delete " + obj.ToString(), obj.Delete);
    }

This is even simpler and doesn't use closures at all.

rleisti2• 17 years ago

Doh!

It would have made more sense if I had some other piece of information to inject in, like if you had a database reference that needed to be passed in to the Delete() method.

Then:

  AddButton("Delete " + obj.ToString(), () => obj.Delete(db));

Vs.

  var d = new DeleteHandler(obj, db);
  AddButton("Delete " + obj.ToString(), d.DoDelete);

EDIT: fixed

barrkel• 17 years ago

I implemented anonymous methods in Delphi, and I faced a similar kind of problem trying to sell them, as it were, to the Delphi community.

What they are: functions with state.

Why they're useful: callbacks where a whole separate method or class would be overkill.

Motivating examples:

* Delegating work to a background thread. Captured variables are an easy way to pass along necessary state.

* A generic benchmarking routine. Benchmarking requires taking a measurement before the code, running the code to be measured possibly multiple times, and taking another measurement after the code. If you can turn the code to be measured into a callback, then the benchmarking code can be written once and reused elsewhere. But having to put all your benchmarks into their own methods, much less worry about state and possibly creating classes and initializing data in constructors, is pretty tedious. Closures avoid this tediousness.

(Yes, I'm conflating several concepts here, anonymous functions and lambda closures, for colloquial and didactic reasons. Nitpicking doesn't buy you much in terms of practicality when learning here.)

sli• 17 years ago

Are they similar in any way to Python generators? They are functions with a state, at least.

jrockway• 17 years ago

Multiple closures can share the same state ("environment"). Consider:

  (let ((i 0))
    (list (lambda () (incf i))
          (lambda () (decf i))
          (lambda () i)))

If you evaluate the first function this form returns ("increment") and then the third form ("get"), the result is 1. If you evaluate the whole form again, you now have another "instance", with its own separate shared i.

In Python, you would probably just make this a class. (But sometimes it's nice to have something equivalent to a class that's automatically built for you, and closures are that something.)

projectileboy• 17 years ago

A closure enables you to keep local state in a functional language. Think of it as an object (a really abused word, I know, but it conveys the idea).

So what are the mechanics? Follow me (I'll use Arc):

Lesson 1) A function can return a function:

  arc> (def adder-factory () [+ 1 _])
  #<procedure: adder-factory>
  arc> (= add-1 (adder-factory))
  #<procedure>
  arc> (add-1 10)
  11

Lesson 2) A created function can hang on to information which you pass in:

  arc> (def adder-factory (c) [+ c _])
  #<procedure: adder-factory>
  arc> (= add-17 (adder-factory 17))
  #<procedure>
  arc> (add-17 13)
  30

Lesson 3) The information a function hangs on to can be mutable local state.

  arc> (def make-point ((o x 0) (o y 0))
            "Constructor for creating point objects"
            (let methods (obj
                    get-x (fn () x)
                    get-y (fn () y)
                    set-x [= x _ ]
                    set-y [= y _ ])
                (fn (method . args)
                    (apply (methods method) args))))
  #<procedure: make-point>
  arc> (= pt (make-point))
  #<procedure>
  arc> (= pt (make-point))
  #<procedure>
  arc> (pt 'get-x)
  0
  arc> (pt 'set-x 17)
  17
  arc> (pt 'get-x)
  17

And that is a "closure". The name comes from the fact that the scope of the created function "closes over" the local variable.

Helpful or noisy?

jerf• 17 years ago

I wrote this to answer that question: http://www.jerf.org/iri/post/2542

In particular, I focus on showing an example that is noticeably more complicated without closures, but also a real case that could conceivably come up.

bgray• 17 years ago

Simple Clojure example:

  user=> (defn add-n [n] #(+ n %))
  #'user/add-n
  user=> (def add-2 (add-n 2))
  #'user/add-2
  user=> (add-2 5)
  7

mtomczak• 17 years ago

They are functions you can make up on the spot. That's usually the extent of my explanation.

Surprisingly, most of the people I talk to assume that the "fancy" parts of closures (such as variable value capture and upvalues) are the way such a construct should work and require no further explanation to use them. People who are surprised by capture (and assume, for instance, that the variable will take on the most recent value assigned to it by outside code instead of "remembering the value when the closure was made") get a more lengthy computer science-esque explanation. But I generally find that explanation to be unhelpful if a person can't see why closures are useful; "An anonymous function with some teeth" is often how people I talk to come to closures first, so it's how I let them think about it until the model is no longer helpful.

DanielStraight• 17 years ago

Right. That's exactly how it was with me when I first encountered closures. Explaining something that would be guessed anyway just makes it seem more complicated than it is.

Ixiaus• 17 years ago

A lexical closure is similar in many ways to the class of an object-oriented language. In fact, lexical closures can be used to create an object system.

A class in PHP, for example, can have any number of class variables and for this demonstration consider these variables to be "private" (not accessible outside of the class or its own instances). One can, however, write methods or functions in the class that can provide an interface to operate on or access these private class variables.

That is, essentially, what a lexical closure is but without the overhead of an object system (remember, an object system can be created using closures!). They are a remarkably useful pattern of encapsulating certain kinds of logic.

Continuations, are another fun topic related to lexical closures too...

ericbb• 17 years ago

Closures are different from objects in only a couple of ways: (1) the creation protocol is extremely concise (lambda) (2) the access interface is the function call

Say that closures lower the syntactic overhead of accessing the same flexibility he normally turns to objects for.

So if he uses closures he will effectively use objects in more situations simply because it becomes easier and this will increase the flexibility of his code.

dasil003• 17 years ago

I wrote an article about the closure aspect of Ruby blocks that was intended for beginners:

http://darwinweb.net/articles/41-ruby_blocks_as_closures

It really focuses on just one aspect of closures that Ruby's syntax makes especially convenient, which is that they enable you to write generic functions that don't need to know a single detail about what kind of data they are being applied to.

mping• 17 years ago

Imagine you need a task to be done. Normally, you go and do it yourself. With a closure, you write the instructions and just hand it off to someone :)

zackattack• 17 years ago

Closures are functions that exist inside a variable. Therefore an entire different level of scope, now exists inside that variable. The variable then becomes like an object in that respect.

Easy.

bobbyi• 17 years ago

If they don't already understand object-oriented programming, you make sure they understand that first.

If they understand objects, you explain that closures are the same as objects. They are bundles of state that carry around functions that operate on that state. The only real difference is that a single closure only contains a single function (method) that operates on its data, where a single object can have several methods.

andreyf5• 17 years ago

You have about 10 seconds to get his interest before he starts thinking about his WoW raid tonight, and about 5 minutes for the whole explanation.

Not going to happen. First, you need a person that wants to better themselves as a programmer, and understanding closures is just a part of that. Discounting the time it takes to put someone in that state of mind, closures can be explained in under a minute (edit: OK, maybe two minutes ;)).

A closure is just a (function, bindings) pair. If someone has a Java/C++ background, it might help to think of a closure as an object which has some code with references to variables defined outside of that code. Those variables are dependencies without which you can't run the code.

JavaScript example:

    function foo() {
        var x = 1;
        function bar() {
            return x + 1;
        }
        return bar;
    }

What does foo() return? It returns bar. But it can't just return bar, since bar has a dependency on x, so it returns bar and everything bar references: the closure (bar, { x: 1 }). In practice, you call a closure as you would call a function, but underneath the abstraction, you need to realize what's returned is both bar and the value of x.

csbrooks3• 17 years ago

So when does "x" get evaluated? What if x was a global variable instead? (Honest questions here, not trying to sidetrack the discussion. No, I don't "get" closures.)

btilly3• 17 years ago

Every time you call bar, x gets evaluated again. If x was a global variable, the same thing would happen.

Closures don't change the scope of any existing thing. What they do is force the program to keep scopes around because something still refers to them. Those scopes become a place to stick private data. If you create multiple functions in that scope, they can communicate through that private data. Here is a simple example in JavaScript.

  function closure_demo (x) {
      return {
           set: function (s) {x = s}
         , get: function () {return x}
      };
  }

  var foo = closure_demo("hello");
  var bar = closure_demo("world");

  alert("Bar is " + bar.get());
  alert("Foo is " + foo.get());

  foo.set("goodbye");
  bar.set("earthlings");

  alert("Bar is now " + bar.get());
  alert("Foo is now " + foo.get());

What is happening here? Each function call has a scope. We return an object with 2 functions that can access this scope. Each time you call the function you get a separate scope, so if you call it twice you get different closures. The scope must exist as long as those functions exist.

Shorel• 17 years ago

If x was a global variable then you don't have a closure, just an anonymous function.

Closures require closed over variables.

andreyf5• 17 years ago

Great question! Every time you call foo, a new scope containing x is created. That scope "travels" with bar, and is evaluated (dereferenced might be a better word) just as any other variable inside bar would be - during the execution of bar. A global variable is just a variable in the topmost ("global") scope.

But my example doesn't make the distinction to clarify that at all. Here's a better one:

    function colors() {
        var cs = ['red', 'green', 'blue'],
            i = 0;

        function nextcolor() {
            i = (i+1) % cs.length;
            return cs[i];
        }

        return nextcolor;
    }

Every time you call colors, a new local scope is created. Easiest way to think of scopes is as maps from strings to values. In this case, a new scope { cs: ['red', 'green', 'blue'], i: 0 } is created every time you calls colors. If the language didn't support closures, that scope would be destroyed when colors finishes execution, but in JS, that scope is bound to nextcolor and returned along with nextcolor in a closure. Now, every time you call a particular nextcolor returned by a call to colors, the cs and i variables will reference those same values. Example using firebug:

    >>> A = colors()
    >>> B = colors()
    >>> A() => "green"
    >>> A() => "blue"
    >>> B() => "green"
    >>> B() => "blue"
    >>> A() => "red"
    >>> A() => "green"
    ...

Keep asking if anything is unclear. I was wrong about this taking 1 minute, I suppose... :)

algorias3• 17 years ago

Closures are the context in which a function executes. The best way to demonstrate this is with a silly example that sticks in the mind:

  def learn_how_to_greet(formal=True):
    if formal:
      string = "Good day, Mr. {last_name}."
    else:
      string = "What's up, {first_name}?"
    def greet(**kwds):
      return string.format(**kwds)
    return greet

csbrooks3• 17 years ago

What happens if you do:

  def learn_how_to_greet(formal=True):
    if formal:
      string = "Good day, Mr. {last_name}."
    else:
      string = "What's up, {first_name}?"
    def greet(**kwds):
      return string.format(**kwds)
    ret = greet
    string = "WHOA! Look at me!"
    return ret

In other words, at what point do the variables used in the "greet" function get saved off? And what is saved, the variable name itself, or the data in it?

pufuwozu• 17 years ago

  >>> learn_how_to_greet()(first_name="Joe", last_name="Smith")
  'WHOA! Look at me!'

dazmax• 17 years ago

It should be saved as its value at the point of greet's definition, but Python appears to not have real closures, so the change in the value of string afterwards changes it in greet too.

algorias3• 17 years ago

I think it happens when the closing scope stops being local (i.e. at the time the defined function gets returned).

maximilian• 17 years ago

I've got the following matlab code:

  eta1=0.7;
  omega=10;
  f = @(t) eta1*sin(omega*t) + (1-eta1)*sin(2*omega*t)

I can call

  f(2)

and return a scaler value. I could just make a function in a separate document, but then I would have to pass in my parameters eta1 and omega. By using a closure, I just have to set those parameters in my code beforehand. Also, if I want to change my forcing term, by adding new parameters, I don't have to change the function definition, I just add those parameters right in.

dasht• 17 years ago

I'm going to assume you mean closures as in Scheme, not closures in a pure functional language.

For that, what I would try is an operational approach:

"Joe, you know how local variables are allocated on a stack?" If he says "no" I'm not sure what the next move is but if he says yes, get him to briefly explain it back to you. E.g., get him to say, at least in vague terms, that on entry to some function the local variables get "pushed on the stack".

"Ok, good. So, what happens when the function returns?" Hopefully he can come up with "the variables are popped off the stack".

"Right. Now, how does the program find the variables on the stack?" Hopefully, Joe can come up with the idea of a stack pointer.

"Cool. [Draw a little picture] So, the stack pointer is kind of like a pointer to a structure. The variables are the fields of the structure. See?" Hopefully Joe says "yes".

"Here's an alternative: instead of using a stack, we could allocate that structure just like any other structure. Instead of popping things off the stack at the end, we could just free/delete/whatever the structure. Make sense?" Hopefully he's still with you.

"Only, we don't actually have to free/delete the structure right away. Suppose I write a function inside this other function, like this [write pseudo-code for an anonymous function with some free variables bound to locals. Have the containing function return this anonymous function.]"

"You might think that when we return a 'function pointer' that we're just returning a pointer to the code. But in this case, our 'function pointer' is really two pointers: a pointer to the code and another pointer to the structure containing the local variables it refers to. So now, we DON'T free/delete that structure of local variables when returning. It sticks around. It's used by this 'function pointer' where returning. When we call that function pointer, it gets passed a sekrit hidden magic parameter which is a pointer to that structure. So the function can still keep using those variables." Hopefully Joe is still with you but ask some questions to find out.

"It's a little bit like that structure of local variables is an object, but an object with only one 'method' and that one method is the nested function we wrote. Get it?"

I think that should get a typical Joe (at least one who has some clue about stacks and stack pointers) to a first-level, rough and ready, operational understanding of closures. If it does, you can mention that actual optimized implementations tend to be a lot hairier, that GC is important, that there are deep connections to lambda calculus, etc.

Again, if on the other hand Joe doesn't even have the slightest awareness of an operational model of how a stack-based language works - I'm not sure what you can do for him.

Ask HN: How do you explain closures?

Related Stories