The Problem with Implicit Scoping in CoffeeScript

http://lucumr.pocoo.org/2011/12/22/implicit-scoping-in-coffeescript/

by michaelty • 15 years ago

133 134 15 years ago

lucumr.pocoo.org

rayiner2• 15 years ago

Scheme got this right in 1970. There is no excuse to design a new language this way.

tikhonj• 15 years ago

I agree with this wholeheartedly--Scheme use an extremely simple scoping model that is nonetheless more expressive than Python (before 3, I guess) and CoffeeScript's. In fact, I only really completely understood JavaScript's model--and realized that, even if a little awkward, it was fundamentally elegant--after writing a Scheme interpreter.

wisty• 15 years ago

I think Python's is deliberately unexpressive, forcing you to use local variables pretty much everywhere. This tends to decouple stuff, which is usually good. The model is "everything you do is local, unless you know better, and want to jump through a lot of hoops". Coffeescript works a similar way.

Javascript seems to have the model "everything you do is global, unless you know better".

I'm not a big Javascript hater, but this is one very sore point.

abecedarius• 15 years ago

Scheme dates to about 1975. But there were older languages with this kind of scoping, like John Reynolds' Gedanken and Landin's ISWIM. (I guess I wouldn't count Algol-60 since it was call-by-name.)

showell3018• 15 years ago

CoffeeScript's approach toward top-level variables is quite elegant and simple. When you declare a variable at top-level scope, it is equally available to all code within that file for both reading and writing, with no strange "nonlocal" or ":=" syntax to complicate manners.

Once you understand the reach of CoffeeScript's top-level variables, it is easy to write bug-free code. Since you know that top-level variables have wide scope, you simply need to be judicious about putting variables at top-level scope. If a variable is not needed at top level scope, don't put it there.

the_mitsuhiko6• 15 years ago

Way to miss the issue. I was not suggesting ":=" for global variables at all. I was suggesting ":=" as a replacement for the nonlocal keyword that Python 3 has to solve the issue I was demonstrating.

showell3018• 15 years ago

Nope, I get it, you were suggesting ":=" as a replacement for the "nonlocal" keyword in Python. I'm not sure how my comments demonstrate any lack of reading comprehension. All I said about ":=" and "nonlocal" is that they are overly complicated once you accept that top-level variables can have wide scope.

Obviously, plenty of folks managed to write bug-free code in Python before "nonlocal" was invented. I'm not saying it's a bad idea, but you can avoid bugs without it.

oinksoft• 15 years ago

@mitsuhiko Not gonna happen ;) Forbidding shadowing altogether is a huge win, and a huge conceptual simplification.

How arrogant! You'd think he'd step back for a second and consider the suggestion, but it sounds like he's on autopilot.

latchkey4• 15 years ago

It sounds like he had 140 characters to succinctly say what he wanted to say.

statictype• 15 years ago

Everyday there seems to be a new post on HN, complete with inflammatory headline, criticizing Coffeescript in someway because it doesn't work in the exact way the author expected.

There are lots of people throwing in their $0.02 on how the language should work without having joined the mailing list or seen any discussions on the thought process behind its features.

I'm not saying the suggestion made isn't reasonable, but I can understand glib replies like this from the author that don't make too much effort to explain his stance more than 140 characters.

showell3018• 15 years ago

In fairness, Armin (the author the blog post) did engage Jeremy (the author of coffeescript) on this issue over twitter. I wish that he would allow comments on his blog. I also wish that he could engage the broader CS community in a proper forum before dissing the language and/or creating a fork of the language. He's overreacting. This whole blog post apparently started because he had a naming collision with "log" in one of his programs. CS does have a mailing list, but most of the action happens via github issues.

locci3• 15 years ago

I think this is a recurring discussion from the dawn of coffeescript. See for example: https://github.com/jashkenas/coffee-script/issues/238

I really don't understand why this isn't being fixed: doesn't global by default break encapsulation? I'm probably missing something, but this is the main reason I haven't tried coffeescript yet.

showell3018• 15 years ago

Just to be completely clear, variables are only at top level scope if you declare them at top-level scope. The variables "x" below are completely encapsulated within f1 and f2. The variables at top-level scope are intentional in the code below--I really do intend f1, f2, and how_many_times_functions_have_been_called to refer to the same entity throughout the file.

  how_many_times_functions_have_been_called = 0

  f1 = ->
    how_many_times_functions_have_been_called += 1 # refers to top-level scope
    console.log x # undefined
    x = 1
    console.log x # 1

  f2 = ->
    f1() # refers to f1 at top-level scope
    how_many_times_functions_have_been_called += 1 # refers to lop-level scope
    console.log x # undefined
    x = 2
    console.log x # 2

  f1() # you can call f1, it's at top-level scope
  f2() # you can call f2, it's at top-level scope
  console.log how_many_times_functions_have_been_called # 3, refers to top-level scope
  console.log x? # false, x does not exist at top_level scope

danso2• 15 years ago

I think Jeremy probably has one of the highest programming-chops-to-arrogance ratios among today's top programmers. It was a tweet, he said what he needed to say. He's been more than willing on HN to step in and explain Coffeescript to critics and offer helpful critiques of competing frameworks (well, in the context of Backbone).

Here's how he introduced CoffeeScript to HN almost exactly two years from today: http://news.ycombinator.com/item?id=1014080

tjholowaychuk11• 15 years ago

"top programmers" don't write javascript and web apps, we're all pretty much noobs

equark• 15 years ago

Don't jump the gun. The reason is that he's already had many discussions about this and has formed an opinion.

See: https://github.com/jashkenas/coffee-script/issues/712 https://github.com/jashkenas/coffee-script/issues/238

Whether he's right is another question. But if you don't like his decision you can use the Coco (https://github.com/satyr/coco) fork which fixes it by introducing := for nonlocal assignment.

barrkel5• 15 years ago

The reasoning basically seems to be "if you misuse this feature (i.e. have top-level symbols), you'll get bugs (like the OP discovered), therefore don't misuse this feature (i.e. keep your top scopes clean), and therefore you won't get bugs, and therefore it's not a problem with CoffeeScript".

It shouldn't take a lot of thought to see why this is a somewhat user-hostile, passive-aggressive approach for a language. If something is ill-advised, the language should actively steer you away from it, not dissuade you with subtle bugs a few thousand lines of code down the road.

I think this scheme would be more workable if sigils or similar conventions of some kind were mandatory for top-level symbols. Then it would be much harder to accidentally wander into this problem. The language he seems to be drawing inspiration from, Ruby, does do this.

whateverer2• 15 years ago

@mitsuhiko Not gonna happen ;) Forbidding shadowing altogether is a huge win, and a huge conceptual simplification.

Poppycock. It doesn't get simpler than

  a. newly bound variables are new
  b. any variable which wasn't bound in the present scope must be bound in a enclosing scope.

If your language makes it much more complicated than First Order Logic (http://cnx.org/content/m12081/latest/), you're doing it wrong.

Of course, it's kind of socially acceptable to get wrong because a lot of language designers didn't think it through and used the same operator for both binding and reassigning a variable (i.e. '=').

jashkenas7• 15 years ago

Sorry -- that was supposed to be a friendly tweet-sized response. The discussion about this "feature" is always good to have, especially because it's one of the most rightfully controversial changes that CoffeeScript makes to JavaScript semantics.

There are dozens of lengthy conversations about this in the CoffeeScript issues, if you'd like to take a deeper look.

latchkey4• 15 years ago

Kind of a side note to the posting, but I just have to say: Please make your usage of parens consistent. If you aren't going to use them, don't use them everywhere.

Here is an example of what I'm talking about:

if isAir cx, cy, cz + 1 then addPlane('near', block)

Should be:

if isAir cx, cy, cz + 1 then addPlane 'near', block

Personally, I use them everywhere because I like having the stronger visual clue that this is a method I'm calling. I think making them optional in CS was a bad idea.

if isAir(cx, cy, cz + 1) then addPlane('near', block)

imho, so much more readable.

MatthewPhillips2• 15 years ago

Not allowing parens would mess up the usage of anonymous functions.

latchkey4• 15 years ago

Not sure I understand you or maybe you don't understand me. I was talking about using parens for method/functional calls, not getting rid of them from CS.

leafo• 15 years ago

Look at how MoonScript handles this: http://moonscript.org/reference/#the_using_clause_controllin...

I didn't want to change the default semantics, but I wanted to have a way for the programmer to be safe if they wanted to, so I created the `using` keyword for function declarations.

You explicitly declare what you intend to overwrite in the lexical scope, including overwriting nothing at all with `using nil`.

showell3018• 15 years ago

FWIW this is how CS works:

  top_level_variable = null

  f = ->
    top_level_variable = "hello"

  f() 
  console.log top_level_variable # prints hello

cheald8• 15 years ago

The concern is that because Coffeescript automatically scopes variables to the scope of their first reference, it can introduce maintenance issues. Consider the following:

    foo = ->
      bar = "woot!"
      console.log bar

This compiles to:

    var foo;
    foo = function() {
      var bar;
      bar = "woot!";
      return console.log(bar);
    };

bar is locally scoped to foo(). Now, 2 weeks later and 200 lines earlier, you come along and define:

    bar = ->
      alert "Holy crap cheese is awesome!"

Which compiles to:

    var bar, foo;
    bar = function() {
      return alert("Holy crap cheese is awesome!");
    };
    foo = function() {
      bar = "woot!";
      return console.log(bar);
    };

Now, all of a sudden, the "bar" reference in foo isn't scoped to foo() anymore, it's scoped globally, and once you invoke foo(), it'll replace the function bar with a string, potentially breaking your app. It's an ease-of-maintenance issue.

This isn't consistent behavior, though. If you define your top-level bar() function after foo, like so:

    foo = ->
      bar = "woot!"
      console.log bar

    bar = ->
      alert "Holy crap cheese is awesome!"

Then you get "correct" scoping (and the outer bar is shadowed):

    var bar, foo;
    foo = function() {
      var bar;
      bar = "woot!";
      return console.log(bar);
    };
    bar = function() {
      return alert("Holy crap cheese is awesome!");
    };

On one hand, it could be argued that this is a "name things better" problem, but on the other, I have to agree that it'd be nice to be able to explicitly scope things when needed. Given that the behaviors are divergent based on what order the variables appear in, I'd say it's confusing enough that a way to explicitly say "hey, I know what I'm doing, I want to shadow any outer variables and declare local scope here" would be useful.

mutewinter• 15 years ago

This is a better explanation of the problem than the blog post. Thanks!

satyr2• 15 years ago

The inconsistency doesn't stop there; `for` variables are specialcased:

    bar = ->
      alert "Holy crap cheese is awesome!"

    foo = ->
      for bar of bars
        console.log bar
      return

↓

    var bar, foo;

    bar = function() {
      return alert("Holy crap cheese is awesome!");
    };

    foo = function() {
      var bar;
      for (bar in bars) {
        console.log(bar);
      }
    };

boundlessdreamz• 15 years ago

Thanks for this! Way better explanation than the blog post :)

Is there anyway to make the scope explicit in coffeescript?

showell3018• 15 years ago

There is some discussion here on why a "bar" variable at top level only affects the scoping of other "bar" variables that are lexically below "bar" in the file, not above it.

https://github.com/jashkenas/coffee-script/issues/1121

tjholowaychuk11• 15 years ago

I like the look of coffeescript's assignment better, but I cant help but think "let" is much less ambiguous, once you see it you look no further. This reminds me a bit of Ruby, where "foo" could be a function or variable potentially from anywhere so it's a little unclear although better looking.

stoodder2• 15 years ago

Why not allow CoffeeScript to use the 'var' keyword, explicitly telling CS that this variable should be scoped locally even though it may be shadowing another variable? This seems consistent with their approach of allowing (although optional) native javascript syntax such as {}, and []. This still allows CS to stick to it's paradigm of forbidding shadowing unless we explicitly state that we know what we're doing.

jashkenas7• 15 years ago

Because we're aiming for a conceptual simplification.

Pretend like you're a beginner, learning this stuff for the first time. If everywhere you see a variable "A", within a certain lexical scope, it means the same thing ... that's much simpler to understand than if "A" means three different things at three different places, because you happened to shadow it twice.

stoodder2• 15 years ago

Yea, I completely get what you're saying and for the most part agree. To be frank, I think the author of the article made an error in deconstructing on the 'Math' object (at least at a global scope). 'Math' provides a namespace for all of its methods and a similar approach should be taken to other libraries/pieces of code. I also agree that keeping things simple and straight forward makes sense, but you're doing it by forcing one to abide by those standards although someone might have completely legitimate reasons for explicitly scoping their variables.

Either way, I think it is what it is and the benefits of CS very much outweigh the cons. Thanks for the feedback

gerggerg• 15 years ago

Considering we won't see this changed since the author has already closed the issue and expressed his satisfaction with the current rules this article should at least serve as a reminder for errors not to repeat with the next language someone designs.

It's open source. Why not fork it and get some like minded coders to change it with you?

scribu• 15 years ago

There already is a fork that changes this (Coco), already mentioned in another comment: http://news.ycombinator.org/item?id=3380423

limeblack• 15 years ago

Another Issue:

Although the following examples could become unambiguous with parenthesis, these examples demonstrates how a trivially overlooked ending delimiter further complicated the language. Not only is the intent of the CoffeScript code unclear in the examples below but the slight variation in the CoffeScript, produces radically different output. The CoffeeScript differences are so small it would be easy for someone to add accidentally while editing. Anonymous function passing and function calling in Javascript require no additional wrappers or edits, while in CoffeeScript you must add special case clarity.

http://img542.imageshack.us/img542/7379/coffeescripttojavasc...

gcv• 15 years ago

It's worth pointing out that JavaScript 1.7 resolves this mess by introducing block scoping using the "let" keyword. It works just like it does in Scheme, Common Lisp, and Clojure (i.e., correctly). Not supported in anything except Firefox, unfortunately.

swannodette4• 15 years ago

... and Standard ML, OCaml, Haskell, Smalltalk, etc

draegtun2• 15 years ago

... and Perl except its called my instead of let

buddydvd• 15 years ago

I found one of the referenced links in Github issue #712 quite interesting:

http://www.rubyist.net/~matz/slides/rc2003/mgp00010.html

Source: https://github.com/jashkenas/coffee-script/issues/712#issuec...

showell3018• 15 years ago

These links explain the thought process behind CS's current behavior:

https://github.com/jashkenas/coffee-script/issues/712#issuec...

perfunctory• 15 years ago

This is my biggest problem with CoffeeScript. And the author stubbornly refuses to fix it. Apparently it's some sort of Ruby religion.

danmaz74• 15 years ago

With "var" and shadowing you can still shoot yourself in the foot, it's just the other foot.

If you need global variables, it's sensible to just adopt a simple naming convention, like prepending g_ (or whatever pleases you) to all your variables. I already did that with plain JS and it's well worth the "effort".

mhansen• 15 years ago

I'll copy below jashkenas' longer answer from the old github issue about this, https://github.com/jashkenas/coffee-script/issues/712#issuec...

"""

Sorry, folks, but I'm afraid I disagree completely with this line of reasoning -- let me explain why:

Making assignment and declaration two different "things" is a huge mistake. It leads to the unexpected global problem in JavaScript, makes your code more verbose, is a huge source of confusion for beginners who don't understand well what the difference is, and is completely unnecessary in a language. As an existence proof, Ruby gets along just fine without it.

However, if you're not used to having a language without declarations, it seems scary, for the reasons outlined above: "what if someone uses my variable at the top of the file?". In reality, it's not a problem. Only the local variables in the current file can possibly be in scope, and well-factored code has very few variables in the top-level scope -- and they're all things like namespaces and class names, nothing that risks a clash.

And if they do clash, shadowing the variable is the wrong answer. It completely prevents you from making use of the original value for the remainder of the current scope. Shadowing doesn't fit well in languages with closures-by-default ... if you've closed over that variable, then you should always be able to refer to it.

The real solution to this is to keep your top-level scopes clean, and be aware of what's in your lexical scope. If you're creating a variable that's actually a different thing, you should give it a different name.

Closing as a wontfix, but this conversation is good to have on the record.

"""

the_mitsuhiko6• 15 years ago

> As an existence proof, Ruby gets along just fine without it.

Missing that Ruby stops scoping variables at a method and uses separate lexical scoping rules for constants thereby avoiding this issue mostly.

cheald8• 15 years ago

Also, Ruby can shadow method names with local variable names just fine:

    class Foo
      def bar
        "bar"
      end

      def baz
        bar = "foo"
        puts bar
      end

      def bang
        puts bar
      end
    end

    foo = Foo.new
    foo.baz  # => "foo"
    foo.bang # => "bar"

swannodette4• 15 years ago

and well-factored code has very few variables in the top-level scope -- and they're all things like namespaces and class names

Unless you happen to embrace JavaScript's functional side and write lots of top level helper functions (in a closure of course after which you export)

shadowing the variable is the wrong answer. It completely prevents you from making use of the original value for the remainder of the current scope.

This smells of static enforcement - strange for a language expounding JavaScript's dynamic nature and an odd departure from "it's just JavaScript".

Shadowing doesn't fit well in languages with closures-by-default

Not sure what evidence this is based on given the heap of great languages with closures-by-default that give programmers more control over scope without introducing goofy constructs like special assignment operators or global/nonlocal keywords.

CoffeeScript breaks the one real form of encapsulation (which includes the power of naming) that JavaScript has - function locals.

jashkenas7• 15 years ago

I may just ask you in a few minutes when you get in ... but what exactly is the scoping scheme (and generated JavaScript) that you're proposing here, without "var", "nonlocal", ":=", and with shadowing?

In addition, to repeat myself elsewhere in this thread, the goals here are conceptual simplification and readability, not giving the programmer more control over scope. The final result is that hopefully:

    someVariable
      ... more code here ...
        someVariable
          ... more code here ...
            someVariable
          ... more code here ...
        someVariable

... in the above code, you can know that "someVariable" always refers to the same thing. With "var", the above code could allow "someVariable" to refer to three different things, each for slightly different sections of the above chunk of code.

If you really want three different values, use three different names. In all cases, it will read better than shadowing would have.

tjholowaychuk11• 15 years ago

ruby is terribly ambiguous..

shasta• 15 years ago

If you really favor protection from errors, then the correct choice is to require 'var' for new declarations but forbid shadowing (it's an error). Shadowing a la Scheme is a source of errors, too -- you can intend the outer variable, failing to notice it's been shadowed. Of course, if you really favor early detection of errors, you'll also have static name binding.

pwpwp2• 15 years ago

[Shadowing] prevents you from making use of the original value for the remainder of the current scope.

That's totally false.

In languages like Scheme, O'Caml, etc you are never prevented from using the original value.

The point is that lexical, aka static scope is all about lexical pieces of code that you fully control, and all their properties are statically apparent, by looking at a single piece of code.

Scheme:

  (DEFINE FOO 1)
  (LET ((FOO 2)) ... FOO IS SHADOWED HERE ...)

Inside the LET, FOO is shadowed (FOO is "your FOO") and that's lexically, statically apparent by looking at the piece of code.

If you don't want it shadowed, and use the global FOO, you just use another variable name. You cannot be prevented from using the original global FOO, because you choose the local variable names you use in a piece of code.

apgwoz• 15 years ago

You cannot be prevented from using the original global FOO, because you choose the local variable names you use in a piece of code.

Unless you're using a system in which non-hygienic macros are present and they expand into it...

cvshepherd• 15 years ago

> The simple solution is to either add a nonlocal keyword like Python has or to introduce a := parameter that works like = but explicitly overrides a higher level variable.

I disagree. The simple solution to this is to write tests.

Deestan• 15 years ago

> I disagree. The simple solution to this is to write tests.

That's a step backwards. Code error checking should be done as early as possible. In order of earliness:

* Typing in the code. (Ideal: it is clear from the syntax that the code performs X instead of Y.)

* Compiling. (Strong type checking ensures you cannot return 5.3e7 or null from GetHostName.)

* Running the code at all. (Code contracts and assertions trigger if GetHostName returns "".)

* Automated unit tests. (Check that DB.GetHostName() returns the same string given to DB.Connect().)

* Automated integration tests. (Check that the DB module can connect to and retrieve useful data from a dummy database.)

* QA ("Hey Joe, the system hangs when I give "¤;\@" as my username and press the connect button rapidly for a few seconds.")

* Customer ("Hi the system has a problem, please fix.")

The further up, the faster, more accurately and with less "noise" the error can be discovered.

shaunxcode• 15 years ago

Here is a ghetto "let" form in coffee

((a = 5, b = 6, log = x -> console.log x) -> log a + b)()

The Problem with Implicit Scoping in CoffeeScript

Related Stories